Nvidia NIM インテグレーションのメトリクス
Nvidia NIM の監視に使う Datadog の公式インテグレーションです。収集するメトリクスは 34 件あります。
公式の紹介文 (英語)
NVIDIA NIM integration with Datadog enables real-time GPU observability by collecting Prometheus metrics for monitoring.
種別の内訳
- count (集計間隔内の発生回数): 24 件
- gauge (その時点の値): 10 件
メトリクスの一覧
説明は公式の定義の英文をそのまま載せています。
| メトリクス | 種別 | 単位 |
|---|---|---|
nvidia_nim.The observations of end to end request latency bucketed by seconds. | count | - |
nvidia_nim.The total number of observations of end to end request latency. | count | - |
nvidia_nim.The sum of end to end request latency in seconds. | count | second |
nvidia_nim.Number of generation tokens processed. | count | token |
nvidia_nim.GPU KV-cache usage. 1 means 100 percent usage. | gauge | fraction |
nvidia_nim.The max number of concurrently running requests. | gauge | request |
nvidia_nim.Number of requests currently running on GPU. | gauge | request |
nvidia_nim.Number of requests waiting. | gauge | request |
nvidia_nim.Total user and system CPU time spent in seconds. | count | second |
nvidia_nim.Maximum number of open file descriptors. | gauge | file |
nvidia_nim.Number of open file descriptors. | gauge | file |
nvidia_nim.Resident memory size in bytes. | gauge | byte |
nvidia_nim.Time in seconds since process started. | gauge | second |
nvidia_nim.Virtual memory size in bytes. | gauge | byte |
nvidia_nim.Number of prefill tokens processed. | count | token |
nvidia_nim.Number of times this generation was collected. | count | - |
nvidia_nim.Objects collected during GC. | count | - |
nvidia_nim.Uncollectable objects found during GC. | count | - |
nvidia_nim.Python platform information. | gauge | - |
nvidia_nim.The count of failed requests. | count | request |
nvidia_nim.The count of finished requests. | count | request |
nvidia_nim.Number of generation tokens processed. | count | - |
nvidia_nim.Number of generation tokens processed. | count | - |
nvidia_nim.Number of generation tokens processed. | count | token |
nvidia_nim.Number of prefill tokens processed. | count | - |
nvidia_nim.Number of prefill tokens processed. | count | - |
nvidia_nim.Number of prefill tokens processed. | count | token |
nvidia_nim.Count of successfully processed requests. | count | - |
nvidia_nim.The observations of time per output token bucketed by seconds. | count | - |
nvidia_nim.The total number of observations of time per output token. | count | - |
nvidia_nim.The sum of time per output token in seconds. | count | second |
nvidia_nim.The observations of time to first token bucketed by seconds. | count | - |
nvidia_nim.The total number of observations of time to first token. | count | - |
nvidia_nim.The sum of time to first token in seconds. | count | second |
公式ドキュメント
メトリクスの名前と説明は Datadog の公式インテグレーションの定義 (Copyright Datadog, Inc.・BSD-3-Clause) を英文のまま載せています。利用条件の全文は サイトについてにあります。