Flink integration metrics
The official Datadog integration for monitoring Flink. It collects 87 metrics.
Official summary
Track metrics for your flink jobs.
Metric types
- gauge (value at a point in time): 58
- count (occurrences in the interval): 29
Metrics
Descriptions are the official definitions, unedited.
| Metric | Type | Unit |
|---|---|---|
flink.The recent CPU usage of the JVM in the jobmanager | gauge | percent |
flink.The CPU time used by the JVM in the jobmanager | gauge | second |
flink.The total number of classes loaded since the start of the JVM in the jobmanager | count | - |
flink.The total number of classes unloaded since the start of the JVM in the jobmanager | count | - |
flink.The number of buffers in the direct buffer pool in the jobmanager | count | buffer |
flink.The amount of memory used by the JVM for the direct buffer pool in the jobmanager | gauge | byte |
flink.The total capacity of all buffers in the direct buffer pool in the jobmanager | count | byte |
flink.The amount of heap memory guaranteed to be available to the JVM in the jobmanager | gauge | byte |
flink.The maximum amount of heap memory that can be used for memory management in the jobmanager | gauge | byte |
flink.The amount of heap memory currently used in the jobmanager | gauge | byte |
flink.The number of buffers in the mapped buffer pool in the jobmanager | gauge | buffer |
flink.The amount of memory used by the JVM for the mapped buffer pool in the jobmanager | gauge | byte |
flink.The total capacity of all buffers in the mapped buffer pool in the jobmanager | count | byte |
flink.The amount of non-heap memory guaranteed to be available to the JVM in the jobmanager | gauge | byte |
flink.The maximum amount of non-heap memory that can be used for memory management in the jobmanager | gauge | byte |
flink.The amount of non-heap memory currently used in the jobmanager | gauge | byte |
flink.The total number of live threads in the jobmanager | count | thread |
flink.For jobs currently in a failing/recovering situation- the time elapsed during this outage. Returns 0 for running jobs and -1 for completed jobs | gauge | millisecond |
flink.The number of buffered bytes during alignment over all subtasks for the last checkpoint | gauge | byte |
flink.The time it took to complete the last checkpoint | gauge | millisecond |
flink.The path where the last external checkpoint was stored | gauge | - |
flink.Timestamp when the last checkpoint was restored at the coordinator | gauge | millisecond |
flink.The total size of the last checkpoint | gauge | byte |
flink.The total number of restarts since this job was submitted, including full restarts and fine-grained restarts | gauge | - |
flink.The number of successfully completed checkpoints | count | - |
flink.The number of failed checkpoints | count | - |
flink.The number of in progress checkpoints | gauge | - |
flink.The time it took to restart the job or how long the current restart has been in progress | gauge | millisecond |
flink.The number of total checkpoints (in progress completed and failed) | count | - |
flink.The time that the job has been running without interruption. Returns -1 for completed jobs | gauge | millisecond |
flink.The number of registered taskmanagers | gauge | - |
flink.The number of running jobs | gauge | job |
flink.The total number of task slots | gauge | - |
flink.The total number of offset commit failures to Kafka if offset committing is turned on and checkpointing is enabled. Note that committing offsets back to Kafka is only a means to expose consumer progress so a commit failure does not affect the integrity of Flink's checkpointed partition offsets | count | commit |
flink.The total number of successful offset commits to Kafka if offset committing is turned on and checkpointing is enabled | count | commit |
flink.The last watermark this operator has received in its first input. Only for operators with 2 inputs | gauge | millisecond |
flink.The last watermark this operator has received in its second input. Only for operators with 2 inputs | gauge | millisecond |
flink.The last watermark this operator has received. For tasks with 2 inputs this is the minimum of the last received watermarks | gauge | millisecond |
flink.The last watermark this operator has emitted | gauge | millisecond |
flink.The number of records this operator has dropped due to arriving late | count | record |
flink.The total number of records this operator has received | count | record |
flink.The number of records this operator receives per second | gauge | record/ |
flink.The total number of records this operator has emitted | count | record |
flink.The total number of records this operator has emitted per second | gauge | record/ |
flink.The total number of InputSplits this data source has processed (if the operator is a data source) | count | - |
flink.An estimate of the input buffers usage | gauge | - |
flink.The number of queued input buffers | gauge | buffer |
flink.The total number of network buffers this task has read from a local source | count | buffer |
flink.The number of network buffers this task reads from a local source per second | gauge | / |
flink.The total number of network buffers this task has read from a remote source | count | buffer |
flink.The number of network buffers this task reads from a remote source per second | gauge | buffer/ |
flink.The total number of bytes this task has read from a local source | count | byte |
flink.The number of bytes this task reads from a local source per second | gauge | byte/ |
flink.The total number of bytes this task has read from a remote source | count | byte |
flink.The number of bytes this task reads from a remote source per second | gauge | byte |
flink.An estimate of the output buffers usage | gauge | - |
flink.The number of queued output buffers | gauge | buffer |
flink.The time in nanoseconds that the last barrier alignment took to complete or how long the current alignment has taken so far | gauge | nanosecond |
flink.The last watermark this task has received. For tasks with 2 inputs this is the minimum of the last received watermarks | gauge | millisecond |
flink.The total number of network buffers this task has emitted | count | buffer |
flink.The number of network buffers this task emits per second | gauge | buffer/ |
flink.The total number of bytes this task has emitted | count | byte |
flink.The number of bytes this task emits per second | gauge | byte/ |
flink.The number of records this task has dropped due to arriving late | count | record |
flink.The total number of records this task has received | count | record |
flink.The number of records this task receives per second | gauge | record/ |
flink.The total number of records this task has emitted | count | record |
flink.The total number of records this task has emitted per second | gauge | record/ |
flink.The recent CPU usage of the JVM in the taskmanager | gauge | percent |
flink.The CPU time used by the JVM in the taskmanager | gauge | second |
flink.The total number of classes loaded since the start of the JVM in the taskmanager | count | - |
flink.The total number of classes unloaded since the start of the JVM in the taskmanager | count | - |
flink.The number of buffers in the direct buffer pool in the taskmanager | gauge | buffer |
flink.The amount of memory used by the JVM for the direct buffer pool in the taskmanager | gauge | byte |
flink.The total capacity of all buffers in the direct buffer pool in the taskmanager | count | byte |
flink.The amount of heap memory guaranteed to be available to the JVM in the taskmanager | gauge | byte |
flink.The maximum amount of heap memory that can be used for memory management in the taskmanager | gauge | byte |
flink.The amount of heap memory currently used in the taskmanager | gauge | byte |
flink.The number of buffers in the mapped buffer pool in the taskmanager | gauge | - |
flink.The amount of memory used by the JVM for the mapped buffer pool in the taskmanager | gauge | byte |
flink.The total capacity of all buffers in the mapped buffer pool in the taskmanager | count | byte |
flink.The amount of non-heap memory guaranteed to be available to the JVM in the taskmanager | gauge | byte |
flink.The maximum amount of non-heap memory that can be used for memory management in the taskmanager | gauge | byte |
flink.The amount of non-heap memory currently used in the taskmanager | gauge | byte |
flink.The total number of live threads in the taskmanager | count | thread |
flink.The number of unused memory segments in the taskmanager | gauge | - |
flink.The number of allocated memory segments in the taskmanager | gauge | - |
Official documentation
Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.