Airflow integration metrics
The official Datadog integration for monitoring Airflow. It collects 97 metrics.
Official summary
Tracks metrics related to DAGs, tasks, pools, executors, etc
Metric types
- count (occurrences in the interval): 49
- gauge (value at a point in time): 48
Metrics
Descriptions are the official definitions, unedited.
| Metric | Type | Unit |
|---|---|---|
airflow.1 if can connect to Airflow, otherwise 0 | count | - |
airflow.Number of non-zero exit code from Celery task. | count | - |
airflow.Number of AirflowTaskTimeout errors raised when publishing Task to Celery Broker. | count | error |
airflow.Milliseconds taken for fetching all Serialized Dags from DB | gauge | millisecond |
airflow.Number of exceptions raised from DAG callbacks. When this happens, it means DAG callback is not working | count | error |
airflow.DAG loading duration in seconds (deprecated) | gauge | second |
airflow.Milliseconds a task spends in the queue (deprecated, use queued_duration). | gauge | millisecond |
airflow.Milliseconds a task spends in the Queued state, before being Running. | gauge | millisecond |
airflow.Milliseconds a task spends in the Scheduled state, before being Queued. | gauge | millisecond |
airflow.Milliseconds taken to finish a task | gauge | millisecond |
airflow.Current duration for ongoing DAG tasks | gauge | second |
airflow.Total number of running tasks | gauge | - |
airflow.Tasks removed from DAG | gauge | second |
airflow.Tasks restored to DAG | gauge | second |
airflow.(DEPRECATED) same behavior as dag_processing.processor_timeouts | gauge | - |
airflow.Number of failures loading any DAG files | count | error |
airflow.Number of DAG files to be considered for the next scan. | gauge | - |
airflow.Number of times we've scanned the filesystem and queued all existing dags. | count | - |
airflow.Number of errors from trying to parse DAG files | gauge | error |
airflow.Milliseconds taken to load the given DAG file | gauge | millisecond |
airflow.Number of queries to Airflow database during parsing per dag_file. | gauge | - |
airflow.Seconds since <dag_file> was last processed | gauge | second |
airflow.Seconds spent processing <dag_file> (in most recent iteration) | gauge | second |
airflow.Number of stalled DagFileProcessorManager | count | - |
airflow.Number of non-SLA callbacks received. | count | - |
airflow.Number of currently running DAG parsing processes | count | - |
airflow.Number of file processors that have been killed due to taking too long | gauge | - |
airflow.Number of SLA callbacks received. | count | - |
airflow.Seconds taken to scan and import all DAG files once | gauge | second |
airflow.DAG bag size | gauge | - |
airflow.Milliseconds taken to check DAG dependencies | gauge | millisecond |
airflow.Milliseconds taken for a DagRun to reach failed state | gauge | millisecond |
airflow.Milliseconds taken for a DagRun to reach success state | gauge | millisecond |
airflow.Milliseconds elapsed between first task start_date and dagrun expected start | gauge | millisecond |
airflow.Milliseconds of delay between the scheduled DagRun start date and the actual DagRun start date | gauge | millisecond |
airflow.Number of datasets marked as orphans because they are no longer referenced in DAG. | gauge | - |
airflow.Number of DAG runs triggered by a dataset update. | count | - |
airflow.Number of updated datasets. | count | - |
airflow.Number of open slots on executor | gauge | - |
airflow.Number of queued tasks on executor | gauge | task |
airflow.Number of running tasks on executor | gauge | task |
airflow.1 if Airflow is healthy, otherwise 0 | count | - |
airflow.Number of ended <job_name> job, ex. SchedulerJob, LocalTaskJob | count | job |
airflow.Number of failed Heartbeats for a <job_name> job, ex. SchedulerJob, LocalTaskJob | count | error |
airflow.Number of started <job_name> job, ex. SchedulerJob, LocalTaskJob | count | job |
airflow.Milliseconds taken to adopt the task instances in Kubernetes Executor. | gauge | millisecond |
airflow.Milliseconds taken for clearing not launched queued tasks in Kubernetes Executor. | gauge | millisecond |
airflow.Number of LocalTaskJob terminations with a return_code. | count | - |
airflow.Operator <operator_name> failures | count | - |
airflow.Operator <operator_name> successes | count | - |
airflow.Number of deferred slots in the pool. | gauge | - |
airflow.Number of open slots in the pool | gauge | - |
airflow.Number of queued slots in the pool | gauge | - |
airflow.Number of running slots in the pool | gauge | - |
airflow.Number of scheduled slots in the pool. | gauge | - |
airflow.Number of starving tasks in the pool | gauge | task |
airflow.Number of used slots in the pool | gauge | - |
airflow.Number of previously succeeded task instances | count | task |
airflow.Count of times a scheduler process tried to get a lock on the critical section (needed to send tasks to the executor) and found it locked by another process. | count | operation |
airflow.Milliseconds spent in the critical section of scheduler loop -- only a single scheduler can enter this loop at a time | gauge | millisecond |
airflow.Milliseconds spent running the critical section task instance query. | gauge | millisecond |
airflow.Number of Orphaned tasks adopted by the Scheduler | count | task |
airflow.Number of Orphaned tasks cleared by the Scheduler | count | task |
airflow.Milliseconds spent running one scheduler loop. | gauge | millisecond |
airflow.Number of tasks that are ready for execution (set to queued) with respect to pool limits, dag concurrency, executor state, and priority. | count | task |
airflow.Number of tasks killed externally | count | task |
airflow.Number of tasks running in executor | count | task |
airflow.Number of tasks that cannot be scheduled because of no open slot in pool | count | task |
airflow.Number of tasks without DagRuns or with DagRuns not in Running state | count | task |
airflow.Scheduler heartbeats | count | - |
airflow.Number of failed SLA miss callback notification attempts. | count | - |
airflow.Number of failed SLA miss email notification attempts | count | task |
airflow.Number of SLA misses. | count | - |
airflow.Number of failures caused by exception in the previous smart sensor poking loop | count | error |
airflow.Number of infrastructure failures in the previous smart sensor poking loop | count | error |
airflow.Number of exceptions in the previous smart sensor poking loop | count | error |
airflow.Number of newly succeeded tasks poked by the smart sensor in the previous poking loop | count | task |
airflow.Number of tasks poked by the smart sensor in the previous poking loop | count | task |
airflow.Percentage of CPU used by a task. | gauge | percent |
airflow.Milliseconds taken to run a task. | gauge | millisecond |
airflow.Task instances created | gauge | second |
airflow.Percentage of memory used by a task. | gauge | percent |
airflow.Milliseconds a task spends in the Queued state, before being Running. | gauge | millisecond |
airflow.Milliseconds a task spends in the Scheduled state, before being Queued. | gauge | millisecond |
airflow.Number of tasks instances created for a given Operator | count | task |
airflow.Number of tasks removed for a given dag (i.e. task no longer exists in DAG) | count | task |
airflow.Number of tasks restored for a given dag (i.e. task instance which was previously in REMOVED state in the DB is added to DAG file) | count | task |
airflow.Number of completed task in a given dag. | count | task |
airflow.Number of started task in a given dag. | count | task |
airflow.Overall task instances failures | count | task |
airflow.Overall task instances successes | count | task |
airflow.Triggerer heartbeats. | count | - |
airflow.Number of triggers that blocked the main thread (likely due to not being async). | count | - |
airflow.Number of triggers that errored before they could fire an event. | count | - |
airflow.Number of triggers currently running for a triggerer (described by hostname). | gauge | - |
airflow.Number of triggers that have fired at least one event. | count | - |
airflow.Zombie tasks killed | count | task |
Official documentation
Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.