Nvidia Triton integration metrics
The official Datadog integration for monitoring Nvidia Triton. It collects 42 metrics.
Official summary
NVIDIA Triton Inference Server is open source inference-serving software
Metric types
- gauge (value at a point in time): 22
- count (occurrences in the interval): 20
Metrics
Descriptions are the official definitions, unedited.
| Metric | Type | Unit |
|---|---|---|
nvidia_triton.Total cache insertion duration, in microseconds | gauge | microsecond |
nvidia_triton.Total cache lookup duration (hit and miss), in microseconds | gauge | microsecond |
nvidia_triton.Number of responses stored in response cache | gauge | - |
nvidia_triton.Number of cache evictions in response cache | gauge | - |
nvidia_triton.Number of cache hits in response cache | gauge | - |
nvidia_triton.Number of cache lookups in response cache | gauge | - |
nvidia_triton.Number of cache misses in response cache | gauge | - |
nvidia_triton.Cache utilization [0.0 - 1.0] | gauge | - |
nvidia_triton.CPU total memory (RAM), in bytes | gauge | byte |
nvidia_triton.CPU used memory (RAM), in bytes | gauge | byte |
nvidia_triton.CPU utilization rate [0.0 - 1.0] | gauge | - |
nvidia_triton.GPU energy consumption in joules since the Triton Server started | count | - |
nvidia_triton.GPU total memory, in bytes | gauge | byte |
nvidia_triton.GPU used memory, in bytes | gauge | byte |
nvidia_triton.GPU power management limit in watts | gauge | watt |
nvidia_triton.GPU power usage in watts | gauge | watt |
nvidia_triton.GPU utilization rate [0.0 - 1.0) | gauge | - |
nvidia_triton.Cumulative compute inference duration in microseconds (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative compute inference duration in microseconds (count) (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative compute inference duration in microseconds (quantile)(does not include cached requests) | gauge | microsecond |
nvidia_triton.Cumulative compute inference duration in microseconds (sum) (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative compute input duration in microseconds (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative compute input duration in microseconds (sum) (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative compute input duration in microseconds (quantile) (does not include cached requests) | gauge | microsecond |
nvidia_triton.Cumulative compute input duration in microseconds (count) (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative inference compute output duration in microseconds (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative inference compute output duration in microseconds (count) (does not include cached requests) | count | microsecond |
nvidia_triton.Cumulative inference compute output duration in microseconds (quantile) (does not include cached requests) | gauge | microsecond |
nvidia_triton.Cumulative inference compute output duration in microseconds (sum) (does not include cached requests) | count | microsecond |
nvidia_triton.Number of inferences performed (does not include cached requests) | count | - |
nvidia_triton.Number of model executions performed (does not include cached requests) | count | - |
nvidia_triton.Instantaneous number of pending requests awaiting execution per-model. | gauge | - |
nvidia_triton.Cumulative inference queuing duration in microseconds (includes cached requests) | count | microsecond |
nvidia_triton.Summary of inference queuing duration in microseconds (count) (includes cached requests) | count | microsecond |
nvidia_triton.Summary of inference queuing duration in microseconds (quantile) (includes cached requests) | gauge | microsecond |
nvidia_triton.Summary of inference queuing duration in microseconds (sum) (includes cached requests) | count | microsecond |
nvidia_triton.Cumulative inference request duration in microseconds (includes cached requests) | count | microsecond |
nvidia_triton.Summary of inference request duration in microseconds (count) (includes cached requests) | count | microsecond |
nvidia_triton.Summary of inference request duration in microseconds (quantile) (includes cached requests) | gauge | microsecond |
nvidia_triton.Summary of inference request duration in microseconds (sum) (includes cached requests) | count | microsecond |
nvidia_triton.Number of failed inference requests, all batch sizes | count | - |
nvidia_triton.Number of successful inference requests, all batch sizes | count | - |
Official documentation
Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.