Nvidia NIM integration metrics

The official Datadog integration for monitoring Nvidia NIM. It collects 34 metrics.

Definitions as of 2026-04-15

Official summary

NVIDIA NIM integration with Datadog enables real-time GPU observability by collecting Prometheus metrics for monitoring.

Metric types

count (occurrences in the interval): 24
gauge (value at a point in time): 10

Metrics

Descriptions are the official definitions, unedited.

Metric	Type	Unit
`nvidia_nim.e2e_request_latency.seconds.bucket`The observations of end to end request latency bucketed by seconds.	count	-
`nvidia_nim.e2e_request_latency.seconds.count`The total number of observations of end to end request latency.	count	-
`nvidia_nim.e2e_request_latency.seconds.sum`The sum of end to end request latency in seconds.	count	second
`nvidia_nim.generation_tokens.count`Number of generation tokens processed.	count	token
`nvidia_nim.gpu_cache_usage_percent`GPU KV-cache usage. 1 means 100 percent usage.	gauge	fraction
`nvidia_nim.num_request.max`The max number of concurrently running requests.	gauge	request
`nvidia_nim.num_requests.running`Number of requests currently running on GPU.	gauge	request
`nvidia_nim.num_requests.waiting`Number of requests waiting.	gauge	request
`nvidia_nim.process.cpu_seconds.count`Total user and system CPU time spent in seconds.	count	second
`nvidia_nim.process.max_fds`Maximum number of open file descriptors.	gauge	file
`nvidia_nim.process.open_fds`Number of open file descriptors.	gauge	file
`nvidia_nim.process.resident_memory_bytes`Resident memory size in bytes.	gauge	byte
`nvidia_nim.process.start_time_seconds`Time in seconds since process started.	gauge	second
`nvidia_nim.process.virtual_memory_bytes`Virtual memory size in bytes.	gauge	byte
`nvidia_nim.prompt_tokens.count`Number of prefill tokens processed.	count	token
`nvidia_nim.python.gc.collections.count`Number of times this generation was collected.	count	-
`nvidia_nim.python.gc.objects.collected.count`Objects collected during GC.	count	-
`nvidia_nim.python.gc.objects.uncollectable.count`Uncollectable objects found during GC.	count	-
`nvidia_nim.python.info`Python platform information.	gauge	-
`nvidia_nim.request.failure.count`The count of failed requests.	count	request
`nvidia_nim.request.finish.count`The count of finished requests.	count	request
`nvidia_nim.request.generation_tokens.bucket`Number of generation tokens processed.	count	-
`nvidia_nim.request.generation_tokens.count`Number of generation tokens processed.	count	-
`nvidia_nim.request.generation_tokens.sum`Number of generation tokens processed.	count	token
`nvidia_nim.request.prompt_tokens.bucket`Number of prefill tokens processed.	count	-
`nvidia_nim.request.prompt_tokens.count`Number of prefill tokens processed.	count	-
`nvidia_nim.request.prompt_tokens.sum`Number of prefill tokens processed.	count	token
`nvidia_nim.request.success.count`Count of successfully processed requests.	count	-
`nvidia_nim.time_per_output_token.seconds.bucket`The observations of time per output token bucketed by seconds.	count	-
`nvidia_nim.time_per_output_token.seconds.count`The total number of observations of time per output token.	count	-
`nvidia_nim.time_per_output_token.seconds.sum`The sum of time per output token in seconds.	count	second
`nvidia_nim.time_to_first_token.seconds.bucket`The observations of time to first token bucketed by seconds.	count	-
`nvidia_nim.time_to_first_token.seconds.count`The total number of observations of time to first token.	count	-
`nvidia_nim.time_to_first_token.seconds.sum`The sum of time to first token in seconds.	count	second

Official documentation

Official documentation for the Nvidia NIM integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Official summary

Metric types

Metrics

Official documentation

Related products

Other AI/ML integrations