TorchServe インテグレーションのメトリクス
TorchServe の監視に使う Datadog の公式インテグレーションです。収集するメトリクスは 33 件あります。
公式の紹介文 (英語)
Monitor the health and performance of TorchServe
種別の内訳
- gauge (その時点の値): 27 件
- count (集計間隔内の発生回数): 6 件
メトリクスの一覧
説明は公式の定義の英文をそのまま載せています。
| メトリクス | 種別 | 単位 |
|---|---|---|
torchserve.Maximum batch size that a model is expected to handle. | gauge | - |
torchserve.Whether or not the model was loaded when TorchServe started. 1 if true, 0 otherwise. | gauge | - |
torchserve.The maximum batch delay time in ms TorchServe waits to receive batch_size number of requests. | gauge | millisecond |
torchserve.Whether or not this version of the model is the default one. 1 if true, 0 otherwise. | gauge | - |
torchserve.Total number of versions for a given model. | gauge | - |
torchserve.Whether or not this worker is using a GPU. 1 if true, 0 otherwise. | gauge | - |
torchserve.Memory used by the worker in byte. | gauge | byte |
torchserve.The status of a given worker. 1 if ready, 2 if loading, 3 if unloading, 0 otherwise. | gauge | - |
torchserve.Current number of workers of a given model. | gauge | - |
torchserve.Maximum number of workers defined of a given model. | gauge | - |
torchserve.Minimum number of workers defined of a given model. | gauge | - |
torchserve.Total number of models. | gauge | - |
torchserve.CPU utilization on host. | gauge | percent |
torchserve.Disk available on host. | gauge | gigabyte |
torchserve.Memory used on host. | gauge | gigabyte |
torchserve.Disk utilization on host. | gauge | percent |
torchserve.GPU memory used on host. | gauge | megabyte |
torchserve.GPU memory utilization on host. | gauge | percent |
torchserve.GPU utilization on host. | gauge | percent |
torchserve.Time spent in backend handler. | gauge | millisecond |
torchserve.Total number of inference requests received. | count | request |
torchserve.Total inference latency in Microseconds. | count | microsecond |
torchserve.Memory available on host. | gauge | megabyte |
torchserve.Memory used on host. | gauge | megabyte |
torchserve.Memory utilization on host. | gauge | percent |
torchserve.Backend prediction time. | gauge | millisecond |
torchserve.Total queue latency in Microseconds. | count | microsecond |
torchserve.Time spent by a job in request queue in Milliseconds. | gauge | millisecond |
torchserve.Total number of requests with response in 200-300 status code range. | count | request |
torchserve.Total number of requests with response in 400-500 status code range. | count | request |
torchserve.Total number of requests with response status code above 500. | count | request |
torchserve.Time taken by worker to load model in Milliseconds. | gauge | millisecond |
torchserve.Time spent in worker thread excluding backend response time in Milliseconds. | gauge | millisecond |
公式ドキュメント
メトリクスの名前と説明は Datadog の公式インテグレーションの定義 (Copyright Datadog, Inc.・BSD-3-Clause) を英文のまま載せています。利用条件の全文は サイトについてにあります。