obsdog

Hudi integration metrics

The official Datadog integration for monitoring Hudi. It collects 33 metrics.

Official summary

Track metrics for your Hudi configuration.

Metric types

  • gauge (value at a point in time): 19
  • rate (per second): 14

Metrics

Descriptions are the official definitions, unedited.

MetricTypeUnit
hudi.action.bytes_writtenThe total amount of bytes written in an action (commit, deltacommit, replacecommit, compaction, etc)ratebyte
hudi.action.commit_timeThe commit time of an action (commit, deltacommit, replacecommit, compaction, etc)gaugemillisecond
hudi.action.compacted_records_updatedThe amount of compacted records updated in an action (commit, deltacommit, replacecommit, compaction, etc)raterecord
hudi.action.create_timeThe creation time of an action (commit, deltacommit, replacecommit, compaction, etc)ratemillisecond
hudi.action.durationThe amount of time it took to successfully perform an action on a batch of records (commit, deltacommit, replacecommit, compaction, etc)gaugemillisecond
hudi.action.files_insertedThe amount of files inserted (commit, deltacommit, replacecommit, compaction, etc)ratefile
hudi.action.files_updatedThe amount of files updated (commit, deltacommit, replacecommit, compaction, etc)ratefile
hudi.action.insert_records_writtenThe number of insert records written in an action (commit, deltacommit, replacecommit, compaction, etc)raterecord
hudi.action.log_files_compactedThe number of log files compacted in an action (commit, deltacommit, replacecommit, compaction, etc)ratefile
hudi.action.log_files_sizeThe size of all the log files in an action (commit, deltacommit, replacecommit, compaction, etc)ratebyte
hudi.action.partitions_writtenThe number of partitions written in an action (commit, deltacommit, replacecommit, compaction, etc)rate-
hudi.action.records_writtenThe number of records written in an action (commit, deltacommit, replacecommit, compaction, etc)raterecord
hudi.action.scan_timeThe total time spent scanned in an action (commit, deltacommit, replacecommit, compaction, etc)ratemillisecond
hudi.action.time.50th_percentileMeasures 50th percentile of time to complete the action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.75th_percentileMeasures 75th percentile of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.95th_percentileMeasures 95th percentile of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.98th_percentileMeasures 98th percentile of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.999th_percentileMeasures 999th percentile of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.99th_percentileMeasures 99th percentile of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.countMeasures count of times to complete an action (commit, deltacommit, replacecommit, compaction, etc)rate/second
hudi.action.time.maxMeasures maximum amount of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.meanMeasures mean amount of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.minMeasures minimum amount of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.time.std_devMeasures standard deviation of time to complete an action (commit, deltacommit, replacecommit, compaction, etc)gaugenanosecond
hudi.action.update_records_writtenThe amount of update records written in an action (commit, deltacommit, replacecommit, compaction, etc)raterecord
hudi.action.upsert_timeThe upsert time of an action (commit, deltacommit, replacecommit, compaction, etc)ratemillisecond
hudi.clean.durationThe total time spent cleaninggaugemillisecond
hudi.clean.files_deletedThe number of files deleted in cleansgaugefile
hudi.finalize.durationThe total time spent finalizinggaugemillisecond
hudi.finalize.files_finalizedThe number of files finalized"gaugefile
hudi.index.command.durationThe time spent performing an index command (UPSERT, INSERT_OVERWRITE, etc)gaugemillisecond
hudi.rollback.durationThe total time spent in rollbackgaugemillisecond
hudi.rollback.files_deletedThe number of files deleted in rollbackgaugefile

Official documentation

Official documentation for the Hudi integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Related products

IntegrationsIntegrations are the bundle of entry points that pour data from external systems such as AWS and MySQL into Datadog. Where you configure them splits into two families, the Agent side and the cloud account side, and which one you choose changes both what you can see and how billing is counted.Foundation and data collectionLog ManagementLog Management is the product that gathers logs from servers, containers, and cloud services in one place and connects them to search, retention, and notification. It has a distinctive structure in which ingestion and indexing are billed separately.Log managementDatadog AgentSoftware that runs resident on the servers and containers you monitor and delivers metrics, logs, and traces to Datadog.Foundation and data collectionMetricsFrom CPU utilization to user sign-ups, metrics are the core data of Datadog, recording every number in your environment as a time series. If you understand how they are counted before how they are collected, you avoid aggregation mistakes and unexpected charges.Foundation and data collection

Other Log Collection integrations

Hugging Face TGI54 metricsIBM ACE151 metricsIBM Db249 metricsIBM MQ94 metricsIBM Spectrum LSF163 metricsIBM WAS53 metricsiboss113 metricsignite195 metricsIIS25 metricsImpala415 metricsIstio482 metricsJBoss/WildFly31 metrics