SGLang インテグレーションのメトリクス

SGLang の監視に使う Datadog の公式インテグレーションです。収集するメトリクスは 99 件あります。

2026-09-30 時点の定義

公式の紹介文 (英語)

Monitor SGLang request performance, token throughput, and KV cache efficiency

AI と機械学習

種別の内訳

gauge (その時点の値): 57 件
count (集計間隔内の発生回数): 42 件

メトリクスの一覧

説明は公式の定義の英文をそのまま載せています。

メトリクス	種別	単位
`sglang.cache_config_info`Cache configuration information.	gauge	-
`sglang.cached_tokens.count`Total number of cached prompt tokens.	count	token
`sglang.cuda_graph.active`Whether CUDA graph execution is active.	gauge	-
`sglang.cuda_graph.passes.count`Total number of CUDA graph execution passes.	count	-
`sglang.decode.sequence_length_sum`Sum of sequence lengths currently being decoded.	gauge	token
`sglang.disaggregation.bootstrap_failures.count`Total number of failed bootstrap requests.	count	-
`sglang.disaggregation.decode_preallocation_queue`Requests waiting for decode preallocation.	gauge	-
`sglang.disaggregation.decode_transfer_queue`Requests waiting for decode transfer.	gauge	-
`sglang.disaggregation.kv_transfer.allocation_milliseconds`Time spent allocating memory for KV transfers.	gauge	millisecond
`sglang.disaggregation.kv_transfer.bootstrap_milliseconds`Time spent bootstrapping KV transfers.	gauge	millisecond
`sglang.disaggregation.kv_transfer.gigabytes_per_second`KV cache transfer throughput.	gauge	gigabyte/second
`sglang.disaggregation.kv_transfer.latency_milliseconds`KV cache transfer latency.	gauge	millisecond
`sglang.disaggregation.kv_transfer.megabytes`KV cache data transferred.	gauge	megabyte
`sglang.disaggregation.prefill_bootstrap_queue`Requests in the prefill bootstrap queue.	gauge	-
`sglang.disaggregation.prefill_inflight_queue`Requests in the prefill inflight queue.	gauge	-
`sglang.disaggregation.prefill_preallocation_queue`Requests waiting for prefill preallocation.	gauge	-
`sglang.disaggregation.prefill_retries.count`Total number of retried prefill requests.	count	-
`sglang.disaggregation.transfer_failures.count`Total number of failed KV transfer requests.	count	-
`sglang.e2e_request_latency.seconds`End-to-end request latency (distribution).	gauge	second
`sglang.e2e_request_latency.seconds.count`End-to-end request latency observation count.	count	-
`sglang.e2e_request_latency.seconds.sum`Sum of end-to-end request latency.	count	second
`sglang.estimated_flops_per_gpu.count`Estimated floating-point operations per GPU.	count	operation
`sglang.estimated_read_bytes_per_gpu.count`Estimated bytes read per GPU.	count	byte
`sglang.estimated_write_bytes_per_gpu.count`Estimated bytes written per GPU.	count	byte
`sglang.forward_execution.seconds.count`Total model forward execution time.	count	second
`sglang.generation_throughput.tokens_per_second`Generation throughput.	gauge	token/second
`sglang.generation_tokens.count`Total number of generation tokens processed.	count	token
`sglang.http.requests.active`Currently active HTTP requests.	gauge	-
`sglang.http.requests.count`Total number of HTTP requests.	count	request
`sglang.http.responses.count`Total number of HTTP responses.	count	response
`sglang.inter_token_latency.seconds`Inter-token latency (distribution).	gauge	second
`sglang.inter_token_latency.seconds.count`Inter-token latency observation count.	count	-
`sglang.inter_token_latency.seconds.sum`Sum of inter-token latency.	count	second
`sglang.kv_cache.full_attention_usage`Full-attention KV cache usage as a fraction.	gauge	fraction
`sglang.kv_cache.hit_rate`Prefix KV cache hit rate.	gauge	fraction
`sglang.kv_cache.max_tokens`Maximum number of tokens in the KV cache pool.	gauge	token
`sglang.kv_cache.pending_preallocation_usage`KV cache usage reserved by pending preallocations.	gauge	fraction
`sglang.kv_cache.swa_usage`Sliding-window-attention KV cache usage as a fraction.	gauge	fraction
`sglang.kv_cache.usage`Overall KV cache usage as a fraction.	gauge	fraction
`sglang.kv_cache.used_tokens`Number of tokens currently stored in the KV cache.	gauge	token
`sglang.mamba_pool.usage`Mamba state pool usage as a fraction.	gauge	fraction
`sglang.num_requests.grammar_waiting`Requests waiting for grammar compilation.	gauge	request
`sglang.num_requests.offline_batch_running`Running low-priority offline batch requests.	gauge	request
`sglang.num_requests.running`Number of running inference requests.	gauge	request
`sglang.num_requests.waiting`Number of inference requests in the waiting queue.	gauge	request
`sglang.process.cpu_seconds.count`Total CPU time consumed by the SGLang process.	count	second
`sglang.process.max_fds`Maximum number of file descriptors.	gauge	-
`sglang.process.open_fds`Number of open file descriptors.	gauge	-
`sglang.process.resident_memory_bytes`Resident memory used by the SGLang process.	gauge	byte
`sglang.process.start_time_seconds`Unix timestamp when the SGLang process started.	gauge	second
`sglang.process.virtual_memory_bytes`Virtual memory used by the SGLang process.	gauge	byte
`sglang.prompt_tokens.count`Total number of prompt tokens processed.	count	token
`sglang.python.gc.collections.count`Python garbage collection runs.	count	-
`sglang.python.gc.objects.collected.count`Python objects collected by garbage collection.	count	object
`sglang.python.gc.objects.uncollectable.count`Python objects that garbage collection could not collect.	count	object
`sglang.python.info`Python runtime information.	gauge	-
`sglang.queue_time.seconds`Time requests spend waiting in a queue (distribution).	gauge	second
`sglang.queue_time.seconds.count`Request queue-time observation count.	count	-
`sglang.queue_time.seconds.sum`Sum of request queue time.	count	second
`sglang.realtime_tokens.count`Tokens processed in real time.	count	token
`sglang.request.generation_tokens`Generation tokens per request (distribution).	gauge	token
`sglang.request.generation_tokens.count`Generation-token histogram observation count.	count	-
`sglang.request.generation_tokens.sum`Sum of generation tokens observed per request.	count	token
`sglang.request.prompt_tokens`Prompt tokens per request (distribution).	gauge	token
`sglang.request.prompt_tokens.count`Prompt-token histogram observation count.	count	-
`sglang.request.prompt_tokens.sum`Sum of prompt tokens observed per request.	count	token
`sglang.request.stage_latency.seconds`Per-stage request latency (distribution).	gauge	second
`sglang.request.stage_latency.seconds.count`Per-stage request latency observation count.	count	-
`sglang.request.stage_latency.seconds.sum`Sum of per-stage request latency.	count	second
`sglang.request.uncached_prompt_tokens`Uncached prompt tokens per request (distribution).	gauge	token
`sglang.request.uncached_prompt_tokens.count`Uncached prompt-token histogram observation count.	count	-
`sglang.request.uncached_prompt_tokens.sum`Sum of uncached prompt tokens observed per request.	count	token
`sglang.requests.aborted.count`Total number of aborted requests.	count	request
`sglang.requests.count`Total number of inference requests processed.	count	request
`sglang.requests.paused`Number of paused requests.	gauge	request
`sglang.requests.retracted`Number of currently retracted requests.	gauge	request
`sglang.requests.retracted_total.count`Total number of retracted requests.	count	request
`sglang.retracted_input_tokens.count`Input tokens belonging to retracted requests.	count	token
`sglang.retracted_output_tokens.count`Output tokens belonging to retracted requests.	count	token
`sglang.routing_keys.active`Number of routing keys with active requests.	gauge	-
`sglang.scheduler.idle.seconds.count`Total time the scheduler spent idle.	count	second
`sglang.scheduler.new_token_ratio`Ratio used by the scheduler when allocating new tokens.	gauge	fraction
`sglang.scheduler.process_cpu.seconds.count`Total scheduler CPU processing time.	count	second
`sglang.scheduler.utilization`Scheduler utilization as a fraction.	gauge	fraction
`sglang.spec_decode.accept_length`Average speculative decoding acceptance length.	gauge	token
`sglang.spec_decode.accept_rate`Average speculative decoding acceptance rate.	gauge	fraction
`sglang.spec_decode.block_accept_length`Average accepted speculative tokens per block.	gauge	token
`sglang.spec_decode.capture_length`Speculative decoding capture length.	gauge	token
`sglang.spec_decode.draft_tokens`Number of draft tokens per speculative decoding step.	gauge	token
`sglang.spec_decode.steps`Number of speculative decoding steps.	gauge	-
`sglang.spec_decode.verify_calls.count`Total speculative decoding verification calls.	count	-
`sglang.startup.cuda_graph.seconds`Time spent capturing CUDA graphs during startup.	gauge	second
`sglang.startup.phase.seconds`Startup duration for the phase identified by the phase tag.	gauge	second
`sglang.startup.seconds`Time taken for the SGLang engine to start.	gauge	second
`sglang.time_to_first_token.seconds`Time from request receipt to first generated token (distribution).	gauge	second
`sglang.time_to_first_token.seconds.count`Time-to-first-token observation count.	count	-
`sglang.time_to_first_token.seconds.sum`Sum of time to first token.	count	second
`sglang.weight_load.seconds`Time taken to load model weights.	gauge	second
`sglang.weight_update.seconds`Duration of the most recent weight update from the source identified by the source tag.	gauge	second

公式ドキュメント

SGLang インテグレーションの公式ドキュメント

メトリクスの名前と説明は Datadog の公式インテグレーションの定義 (Copyright Datadog, Inc.・BSD-3-Clause) を英文のまま載せています。利用条件の全文はサイトについてにあります。

公式の紹介文 (英語)

種別の内訳

メトリクスの一覧

公式ドキュメント

関連する製品

AI と機械学習の他のインテグレーション