obsdog

vLLM インテグレーションのメトリクス

vLLM の監視に使う Datadog の公式インテグレーションです。収集するメトリクスは 62 件あります。

公式の紹介文 (英語)

vLLM is a library for LLM inference and serving

種別の内訳

  • count (集計間隔内の発生回数): 48 件
  • gauge (その時点の値): 14 件

メトリクスの一覧

説明は公式の定義の英文をそのまま載せています。

メトリクス種別単位
vllm.avg.generation_throughput.toks_per_sAverage generation throughput in tokens/sgauge-
vllm.avg.prompt.throughput.toks_per_sAverage prefill throughput in tokens/sgauge-
vllm.cache_config_infoInformation on cache configgauge-
vllm.cpu_cache_usage_percCPU KV-cache usage. 1 means 100 percent usagegaugepercent
vllm.e2e_request_latency.seconds.bucketThe observations of end to end request latency bucketed by seconds.count-
vllm.e2e_request_latency.seconds.countThe total number of observations of end to end request latency.count-
vllm.e2e_request_latency.seconds.sumThe sum of end to end request latency in seconds.countsecond
vllm.estimated_flops_per_gpu.countEstimated floating-point operations per GPU.countoperation
vllm.external_prefix_cache.hits.countExternal prefix cache hits measured in cached tokens.countitem
vllm.external_prefix_cache.queries.countExternal prefix cache queries measured in queried tokens.countitem
vllm.generation_tokens.countNumber of generation tokens processed.count-
vllm.gpu_cache_usage_percGPU KV-cache usage. 1 means 100 percent usagegaugepercent
vllm.num_preemptions.countCumulative number of preemption from the engine.count-
vllm.num_requests.runningNumber of requests currently running on GPU.gauge-
vllm.num_requests.swappedNumber of requests swapped to CPU.gauge-
vllm.num_requests.waitingNumber of requests waiting.gauge-
vllm.process.cpu_seconds.countTotal user and system CPU time spent in seconds.countsecond
vllm.process.max_fdsMaximum number of open file descriptors.gaugefile
vllm.process.open_fdsNumber of open file descriptors.gaugefile
vllm.process.resident_memory_bytesResident memory size in bytes.gaugebyte
vllm.process.start_time_secondsStart time of the process since unix epoch in seconds.gaugesecond
vllm.process.virtual_memory_bytesVirtual memory size in bytes.gaugebyte
vllm.prompt_tokens.cached.countNumber of cached prompt tokens from local and external caches.countitem
vllm.prompt_tokens.countNumber of prefill tokens processed.count-
vllm.python.gc.collections.countNumber of times this generation was collectedcount-
vllm.python.gc.objects.collected.countObjects collected during gccount-
vllm.python.gc.objects.uncollectable.countUncollectable objects found during GCcount-
vllm.python.infoPython platform informationgauge-
vllm.request.decode_time.seconds.bucketRequest decode-time histogram buckets.count-
vllm.request.decode_time.seconds.countNumber of request decode-time observations.count-
vllm.request.decode_time.seconds.sumTotal request decode time.countsecond
vllm.request.generation_tokens.bucketNumber of generation tokens processed.count-
vllm.request.generation_tokens.countNumber of generation tokens processed.count-
vllm.request.generation_tokens.sumNumber of generation tokens processed.count-
vllm.request.params.best_of.bucketHistogram of the best_of request parameter.count-
vllm.request.params.best_of.countHistogram of the best_of request parameter.count-
vllm.request.params.best_of.sumHistogram of the best_of request parameter.count-
vllm.request.params.n.bucketHistogram of the n request parameter.count-
vllm.request.params.n.countHistogram of the n request parameter.count-
vllm.request.params.n.sumHistogram of the n request parameter.count-
vllm.request.prefill_kv_computed_tokens.bucketHistogram buckets for new KV tokens computed during prefill.count-
vllm.request.prefill_kv_computed_tokens.countNumber of prefill KV token observations.count-
vllm.request.prefill_kv_computed_tokens.sumTotal new KV tokens computed during prefill.countitem
vllm.request.prefill_time.seconds.bucketRequest prefill-time histogram buckets.count-
vllm.request.prefill_time.seconds.countNumber of request prefill-time observations.count-
vllm.request.prefill_time.seconds.sumTotal request prefill time.countsecond
vllm.request.prompt_tokens.bucketNumber of prefill tokens processed.count-
vllm.request.prompt_tokens.countNumber of prefill tokens processed.count-
vllm.request.prompt_tokens.sumNumber of prefill tokens processed.count-
vllm.request.queue_time.seconds.bucketRequest queue-time histogram buckets.count-
vllm.request.queue_time.seconds.countNumber of request queue-time observations.count-
vllm.request.queue_time.seconds.sumTotal request queue time.countsecond
vllm.request.success.countCount of successfully processed requests.count-
vllm.spec_decode.num_accepted_tokens.countNumber of speculative decode tokens accepted.countitem
vllm.spec_decode.num_accepted_tokens_per_pos.countNumber of accepted speculative decode tokens by draft position.countitem
vllm.spec_decode.num_draft_tokens.countNumber of speculative decode draft tokens.countitem
vllm.time_per_output_token.seconds.bucketThe observations of time per output token bucketed by seconds.count-
vllm.time_per_output_token.seconds.countThe total number of observations of time per output token.count-
vllm.time_per_output_token.seconds.sumThe sum of time per output token in seconds.countsecond
vllm.time_to_first_token.seconds.bucketThe observations of time to first token bucketed by seconds.count-
vllm.time_to_first_token.seconds.countThe total number of observations of time to first token.count-
vllm.time_to_first_token.seconds.sumThe sum of time to first token in seconds.countsecond

公式ドキュメント

vLLM インテグレーションの公式ドキュメント

メトリクスの名前と説明は Datadog の公式インテグレーションの定義 (Copyright Datadog, Inc.・BSD-3-Clause) を英文のまま載せています。利用条件の全文は サイトについてにあります。

関連する製品

インテグレーションインテグレーションは、AWS や MySQL といった外部システムのデータを Datadog に流し込む「入口の束」です。設定する場所は Agent 側とクラウドアカウント側の 2 系統に分かれ、どちらを選ぶかで見える範囲と課金の数え方が変わります。基盤とデータ収集LLM ObservabilityLLM を組み込んだアプリの 1 リクエストをトレースとして記録し、モデル呼び出しごとの入出力・トークン・遅延・エラーを追い、応答の品質を評価で数値にする機能です。課金の対象はモデル呼び出しを表す LLM スパンだけです。APM とトレーシングログ管理サーバー・コンテナ・クラウドサービスのログを 1 か所に集め、検索・保管・通知につなげる製品です。取り込みとインデックスを分けて課金する独特の構造を持ちます。ログ管理Datadog Agent監視対象のサーバーやコンテナに常駐し、メトリクス・ログ・トレースを Datadog へ送り届けるソフトウェアです。基盤とデータ収集

AI と機械学習の他のインテグレーション

Weaviateメトリクス 101 件Airbyteメトリクス 48 件Anthropic Usage and Costsメトリクス 8 件AWS Inferentia and AWS Trainium Monitoringメトリクス 33 件BentoMLメトリクス 11 件Databricksメトリクス 16 件Nvidia DCGM Exporterメトリクス 45 件Hugging Face TGIメトリクス 54 件Kubeflowメトリクス 48 件LangChainメトリクス 6 件Microsoft Copilotメトリクス 21 件Milvusメトリクス 386 件