Nvidia DCGM Exporter インテグレーションのメトリクス

Nvidia DCGM Exporter の監視に使う Datadog の公式インテグレーションです。収集するメトリクスは 45 件あります。

公式の紹介文 (英語)

Monitors the exposed GPU metrics leveraged by the Nvidia DCGM Exporter

種別の内訳

  • gauge (その時点の値): 37 件
  • count (集計間隔内の発生回数): 8 件

メトリクスの一覧

説明は公式の定義の英文をそのまま載せています。

メトリクス種別単位
dcgm.clock_throttle_reasonsCurrent clock throttle reasons (bitmask of DCGM_CLOCKS_THROTTLE_REASON_*)gauge-
dcgm.correctable_remapped_rows.countNumber of remapped rows for correctable errors.countrow
dcgm.dec_utilizationDecoder utilization (in %).gaugepercent
dcgm.device.countChange in number Devices on the node.countdevice
dcgm.device.totalNumber of Devices on the node.gauge-
dcgm.dram.activeRatio of cycles the device memory interface is active sending or receiving data (in %).gaugefraction
dcgm.enc_utilizationEncoder utilization (in %).gaugepercent
dcgm.fan_speedFan speed for the device in percent 0-100.gaugepercent
dcgm.frame_buffer.freeFree Frame Buffer in MB.gaugemegabyte
dcgm.frame_buffer.reservedReserved Frame Buffer in MB.gaugemegabyte
dcgm.frame_buffer.totalTotal Frame Buffer of the GPU in MB.gaugemegabyte
dcgm.frame_buffer.usedUsed Frame Buffer in MB.gaugemegabyte
dcgm.frame_buffer.used_percentPercentage used of Frame Buffer: Used/(Total - Reserved). Range 0.0-1.0gaugefraction
dcgm.gpu_utilizationGPU utilization (in %).gaugepercent
dcgm.gr_engine_activeRatio of time the graphics engine is active (in %).gaugefraction
dcgm.mem.clockMemory clock frequency (in MHz).gaugemegahertz
dcgm.mem.copy_utilizationMemory utilization (in %).gaugepercent
dcgm.mem.temperatureMemory temperature (in C).gaugedegree celsius
dcgm.nvlink_bandwidth.countChange in number of NVLink bandwidth counters for all lanescount-
dcgm.nvlink_bandwidth.totalTotal number of NVLink bandwidth counters for all lanesgauge-
dcgm.pcie_replay.countChange in number of PCIe retries.count-
dcgm.pcie_replay.totalTotal number of PCIe retries.gauge-
dcgm.pcie_rx_throughput.countChange in PCIe Rx utilization information.count-
dcgm.pcie_rx_throughput.totalPCIe Rx utilization information.gauge-
dcgm.pcie_tx_throughput.countChange PCIe Tx utilization information.count-
dcgm.pcie_tx_throughput.totalPCIe Tx utilization informationgauge-
dcgm.pipe.fp16_activeRatio of cycles the fp16 pipes are active (in %).gaugefraction
dcgm.pipe.fp32_activeRatio of cycles the fp32 pipes are active (in %).gaugefraction
dcgm.pipe.fp64_activeRatio of cycles the fp64 pipes are active (in %).gaugefraction
dcgm.pipe.tensor_activeRatio of cycles the tensor (HMMA) pipe is active (in %).gaugefraction
dcgm.power_management_limitCurrent power limit for the device.gaugewatt
dcgm.power_usagePower draw (in W).gaugewatt
dcgm.pstatePerformance state (P-State) 0-15. 0=highestgauge-
dcgm.row_remap_failureWhether remapping of rows has failed.gauge-
dcgm.slowdown_temperatureSlowdown temperature for the device.gaugedegree celsius
dcgm.sm_activeThe ratio of cycles an SM has at least 1 warp assigned (in %).gaugefraction
dcgm.sm_clockSM clock frequency (in MHz).gaugemegahertz
dcgm.sm_occupancyThe ratio of number of warps resident on an SM (in %).gaugefraction
dcgm.temperatureGPU temperature (in C).gaugedegree celsius
dcgm.total_energy_consumption.countChange in energy consumption (in mJ).countmillijoule
dcgm.total_energy_consumption.totalTotal energy consumption since boot (in mJ)gauge-
dcgm.uncorrectable_remapped_rows.countChange in number of remapped rows for uncorrectable errors.countrow
dcgm.uncorrectable_remapped_rows.totalTotal number of remapped rows for uncorrectable errors.gauge-
dcgm.vgpu_license_statusvGPU License statusgauge-
dcgm.xid_errorsValue of the last XID error encountered.gauge-

公式ドキュメント

Nvidia DCGM Exporter インテグレーションの公式ドキュメント

メトリクスの名前と説明は Datadog の公式インテグレーションの定義 (Copyright Datadog, Inc.・BSD-3-Clause) を英文のまま載せています。利用条件の全文は サイトについてにあります。