LiteLLM integration metrics

The official Datadog integration for monitoring LiteLLM. It collects 152 metrics.

Definitions as of 2026-09-10

Official summary

This integration allows for real-time collection of LiteLLM metrics for enhanced observability and monitoring.

Metrics

Metric types

count (occurrences in the interval): 114
gauge (value at a point in time): 38

Metrics

Descriptions are the official definitions, unedited.

Metric	Type	Unit
`litellm.active_users`Number of billable users in LiteLLM (excludes SCIM-deactivated users)	gauge	user
`litellm.api.key.budget.remaining_hours.metric`Remaining hours for api key budget to be reset	gauge	hour
`litellm.api.key.max_budget.metric`Maximum budget set for api key	gauge	-
`litellm.auth.failed_requests.count`Number of failed requests for auth service in the time period	count	error
`litellm.auth.latency.bucket`Number of observations that fall into each upper_bound latency bucket for auth service	count	-
`litellm.auth.latency.count`Number of latency observations for auth service in the time period	count	-
`litellm.auth.latency.sum`Latency for auth service	count	millisecond
`litellm.auth.total_requests.count`Number of requests for auth service in the time period	count	request
`litellm.batch_write_to_db.failed_requests.count`Number of failed requests for batch_write_to_db service in the time period	count	error
`litellm.batch_write_to_db.latency.bucket`Number of observations that fall into each upper_bound latency bucket for batch_write_to_db service	count	-
`litellm.batch_write_to_db.latency.count`Number of latency observations for batch_write_to_db service in the time period	count	-
`litellm.batch_write_to_db.latency.sum`Latency for batch_write_to_db service	count	millisecond
`litellm.batch_write_to_db.total_requests.count`Number of requests for batch_write_to_db service in the time period	count	request
`litellm.cache.hits.metric.count`Total number of LiteLLM cache hits	count	-
`litellm.cache.misses.metric.count`Total number of LiteLLM cache misses	count	-
`litellm.cached_tokens.metric.count`Total tokens served from LiteLLM cache	count	token
`litellm.callback.logging_failures.metric.count`Total number of failures when emitting logs to callbacks (e.g. s3_v2, langfuse, etc)	count	-
`litellm.check_batch_cost.errors.count`Total number of errors in CheckBatchCost by error type	count	-
`litellm.check_batch_cost.jobs_polled`Number of unprocessed batches found by the last CheckBatchCost poll	gauge	-
`litellm.check_batch_cost.jobs_processed.count`Total number of batches successfully cost-tracked by CheckBatchCost	count	-
`litellm.check_batch_cost.last_run_timestamp`Unix timestamp of the last CheckBatchCost job run	gauge	second
`litellm.deployment.cooled_down.count`Number of times a deployment has been cooled down by LiteLLM load balancing logic in the time period. exception_status is the status of the exception that caused the deployment to be cooled down	count	event
`litellm.deployment.failed_fallbacks.count`Number of failed fallback requests from primary model -> fallback model in the time period	count	error
`litellm.deployment.failure_by_tag_responses.count`Number of failed LLM API calls for a specific LLM deployment by custom metadata tags in the time period	count	error
`litellm.deployment.failure_responses.count`Number of failed LLM API calls for a specific LLM deployment in the time period. exception_status is the status of the exception from the LLM API	count	error
`litellm.deployment.latency_per_output_token.bucket`Number of observations that fall into each upper_bound latency per output token bucket for deployment	count	-
`litellm.deployment.latency_per_output_token.count`Number of latency per output token observations for deployment in the time period	count	-
`litellm.deployment.latency_per_output_token.sum`Latency per output token	count	millisecond
`litellm.deployment.rpm_limit`Deployment RPM limit found in config	gauge	-
`litellm.deployment.state`The state of the deployment: 0 = healthy,1 = partial outage,2 = complete outage	gauge	unit
`litellm.deployment.success_responses.count`Number of successful LLM API calls via litellm in the time period	count	response
`litellm.deployment.successful_fallbacks.count`Number of successful fallback requests from primary model -> fallback model in the time period	count	response
`litellm.deployment.total_requests.count`Number of LLM API calls via litellm in the time period - success + failure	count	request
`litellm.deployment.tpm_limit`Deployment TPM limit found in config	gauge	-
`litellm.guardrail.errors.count`Total number of errors encountered during guardrail execution	count	-
`litellm.guardrail.latency_seconds.bucket`Number of observations that fall into each upper_bound bucket for latency (seconds) for guardrail execution	count	-
`litellm.guardrail.latency_seconds.count`Number of observations for latency (seconds) for guardrail execution in the time period	count	-
`litellm.guardrail.latency_seconds.sum`Latency (seconds) for guardrail execution	count	second
`litellm.guardrail.requests.count`Total number of guardrail invocations	count	-
`litellm.images_generated.metric.count`Number of images generated, from the image generation response	count	-
`litellm.in_flight_requests`Number of HTTP requests currently in-flight on this uvicorn worker	gauge	-
`litellm.in_memory.daily_spend_update_queue.size`Gauge for in_memory_daily_spend_update_queue service	gauge	item
`litellm.in_memory.spend_update_queue.size`Gauge for in_memory_spend_update_queue service	gauge	item
`litellm.input.tokens.count`Number of input tokens from LLM requests in the time period	count	token
`litellm.input_audio_tokens.metric.count`Audio input tokens reported in prompt_tokens_details.audio_tokens	count	token
`litellm.input_cache_creation_tokens.metric.count`Provider-side input tokens written to prompt cache (e.g. Anthropic cache_creation_input_tokens)	count	token
`litellm.input_cached_tokens.metric.count`Provider-side cached input tokens (e.g. OpenAI prompt_tokens_details.cached_tokens, Anthropic cache_read_input_tokens)	count	token
`litellm.input_tokens.metric.count`Total number of input tokens from LLM requests	count	token
`litellm.llm.api.failed_requests.metric.count`Deprecated - use litellm.proxy.failed_requests.metric. Number of failed responses from proxy in the time period - the client did not get a success response from litellm proxy	count	error
`litellm.llm.api.latency.metric.bucket`Number of observations that fall into each upper_bound latency bucket (seconds) for a model's LLM API call	count	-
`litellm.llm.api.latency.metric.count`Number of latency observations (seconds) for a model's LLM API call in the time period	count	-
`litellm.llm.api.latency.metric.sum`Total latency (seconds) for a model's LLM API call	count	second
`litellm.llm.api.time_to_first_token.metric.bucket`Number of observations that fall into each upper_bound time to first token bucket for a model's LLM API call	count	-
`litellm.llm.api.time_to_first_token.metric.count`Number of time to first token observations for a model's LLM API call in the time period	count	-
`litellm.llm.api.time_to_first_token.metric.sum`Time to first token for a model's LLM API call	count	second
`litellm.managed.batch_created.count`Total number of managed batches created	count	-
`litellm.managed.batch_duration_seconds.bucket`Number of observations that fall into each upper_bound bucket for duration of completed managed batches in seconds (completed_at - created_at)	count	-
`litellm.managed.batch_duration_seconds.count`Number of observations for duration of completed managed batches in seconds (completed_at - created_at) in the time period	count	-
`litellm.managed.batch_duration_seconds.sum`Duration of completed managed batches in seconds (completed_at - created_at)	count	second
`litellm.managed.file_created.count`Total number of managed files created	count	-
`litellm.managed.file_deleted.count`Total number of managed file deletions (success or blocked)	count	-
`litellm.managed.file_size_bytes`Size of the most recent managed batch file in bytes (last-seen value per label combination)	gauge	byte
`litellm.mcp.tool_call_spend.metric.count`Total spend on MCP tool calls, segmented by tool and server name	count	-
`litellm.mcp.tool_calls.count`Total MCP tool calls, segmented by tool and server name	count	-
`litellm.org.budget.remaining_hours.metric`Remaining hours for org budget to be reset	gauge	hour
`litellm.org.max_budget.metric`Maximum budget set for org	gauge	-
`litellm.output.tokens.count`Number of output tokens from LLM requests in the time period	count	token
`litellm.output_audio_tokens.metric.count`Audio output tokens reported in completion_tokens_details.audio_tokens	count	token
`litellm.output_reasoning_tokens.metric.count`Reasoning tokens reported in completion_tokens_details.reasoning_tokens	count	token
`litellm.output_tokens.metric.count`Total number of output tokens from LLM requests	count	token
`litellm.overhead_latency.metric.bucket`Number of observations that fall into each upper_bound overhead latency bucket (milliseconds) added by LiteLLM processing	count	-
`litellm.overhead_latency.metric.count`Number of overhead latency observations (milliseconds) added by LiteLLM processing in the time period	count	-
`litellm.overhead_latency.metric.sum`Latency overhead (milliseconds) added by LiteLLM processing	count	millisecond
`litellm.overhead_with_guardrails_latency.metric.bucket`Number of observations that fall into each upper_bound bucket for total internal latency (seconds) added by LiteLLM, including pre/post-call guardrails (excludes the LLM API call)	count	-
`litellm.overhead_with_guardrails_latency.metric.count`Number of observations for total internal latency (seconds) added by LiteLLM, including pre/post-call guardrails (excludes the LLM API call) in the time period	count	-
`litellm.overhead_with_guardrails_latency.metric.sum`Total internal latency (seconds) added by LiteLLM, including pre/post-call guardrails (excludes the LLM API call)	count	second
`litellm.pod_lock_manager.size`Gauge for pod_lock_manager service	gauge	item
`litellm.postgres.failed_requests.count`Number of failed requests for Postgres service in the time period	count	error
`litellm.postgres.latency.bucket`Number of observations that fall into each upper_bound latency bucket for Postgres service	count	-
`litellm.postgres.latency.count`Number of latency observations for Postgres service in the time period	count	-
`litellm.postgres.latency.sum`Latency for Postgres service	count	millisecond
`litellm.postgres.total_requests.count`Number of requests for Postgres service in the time period	count	request
`litellm.process.uptime.seconds`Start time of the process since unix epoch in seconds.	gauge	second
`litellm.provider.remaining_budget.metric`Remaining budget for provider - used when you set provider budget limits	gauge	-
`litellm.provider_cache.creation_input_tokens.metric.count`Total prompt/input tokens written to provider prompt cache (e.g. Anthropic/Bedrock)	count	token
`litellm.provider_cache.read_input_tokens.metric.count`Total prompt/input tokens read from provider prompt cache (e.g. OpenAI/Anthropic/Gemini/Bedrock)	count	token
`litellm.proxy.failed_requests.metric.count`Number of failed responses from proxy in the time period - the client did not get a success response from litellm proxy	count	error
`litellm.proxy.pre_call.failed_requests.count`Number of failed requests for proxy_pre_call service in the time period	count	error
`litellm.proxy.pre_call.latency.bucket`Number of observations that fall into each upper_bound latency bucket for proxy_pre_call service	count	-
`litellm.proxy.pre_call.latency.count`Number of latency observations for proxy_pre_call service in the time period	count	-
`litellm.proxy.pre_call.latency.sum`Latency for proxy_pre_call service	count	millisecond
`litellm.proxy.pre_call.total_requests.count`Number of requests for proxy_pre_call service in the time period	count	request
`litellm.proxy.total_requests.metric.count`Number of requests made to the proxy server in the time period - track number of client side requests	count	request
`litellm.redis.daily_agent_spend_update_queue.size`Number of pending daily agent spend updates in the Redis queue waiting to be written to the database	gauge	item
`litellm.redis.daily_end_user_spend_update_queue.size`Number of pending daily end user spend updates in the Redis queue waiting to be written to the database	gauge	item
`litellm.redis.daily_spend_update_queue.size`Gauge for redis_daily_spend_update_queue service	gauge	item
`litellm.redis.daily_tag_spend_update_queue.failed_requests.count`Number of failed requests for redis_daily_tag_spend_update_queue service in the time period	count	error
`litellm.redis.daily_tag_spend_update_queue.latency.bucket`Number of observations that fall into each upper_bound latency bucket for redis_daily_tag_spend_update_queue service	count	-
`litellm.redis.daily_tag_spend_update_queue.latency.count`Number of latency observations for redis_daily_tag_spend_update_queue service in the time period	count	-
`litellm.redis.daily_tag_spend_update_queue.latency.sum`Latency for redis_daily_tag_spend_update_queue service	count	millisecond
`litellm.redis.daily_tag_spend_update_queue.total_requests.count`Number of requests for redis_daily_tag_spend_update_queue service in the time period	count	request
`litellm.redis.daily_team_spend_update_queue.failed_requests.count`Number of failed requests for redis_daily_team_spend_update_queue service in the time period	count	error
`litellm.redis.daily_team_spend_update_queue.latency.bucket`Number of observations that fall into each upper_bound latency bucket for redis_daily_team_spend_update_queue service	count	-
`litellm.redis.daily_team_spend_update_queue.latency.count`Number of latency observations for redis_daily_team_spend_update_queue service in the time period	count	-
`litellm.redis.daily_team_spend_update_queue.latency.sum`Latency for redis_daily_team_spend_update_queue service	count	millisecond
`litellm.redis.daily_team_spend_update_queue.total_requests.count`Number of requests for redis_daily_team_spend_update_queue service in the time period	count	request
`litellm.redis.failed_requests.count`Number of failed requests for Redis service in the time period	count	error
`litellm.redis.latency.bucket`Number of observations that fall into each upper_bound latency bucket for Redis service	count	-
`litellm.redis.latency.count`Number of latency observations for Redis service in the time period	count	-
`litellm.redis.latency.sum`Total latency (milliseconds) for Redis service	count	millisecond
`litellm.redis.spend_update_queue.size`Gauge for redis_spend_update_queue service	gauge	item
`litellm.redis.total_requests.count`Number of requests for Redis service in the time period	count	request
`litellm.remaining.api_key.budget.metric`Remaining budget for api key	gauge	-
`litellm.remaining.api_key.requests_for_model`Remaining requests API Key can make for model (model based rpm limit on key)	gauge	request
`litellm.remaining.api_key.tokens_for_model`Remaining tokens API Key can make for model (model based tpm limit on key)	gauge	token
`litellm.remaining.org_budget.metric`Remaining budget for org	gauge	-
`litellm.remaining.requests`Remaining requests for model, returned from LLM API Provider	gauge	request
`litellm.remaining.team_budget.metric`Remaining budget for team	gauge	-
`litellm.remaining.user_budget.metric`Remaining budget for user	gauge	-
`litellm.remaining_requests.metric`Track x-ratelimit-remaining-requests returned from LLM API Deployment	gauge	request
`litellm.remaining_tokens`Remaining tokens for model, returned from LLM API Provider	gauge	token
`litellm.remaining_tokens.metric`Remaining tokens for model, returned from LLM API Provider	gauge	token
`litellm.request.queue_time_seconds.bucket`Number of observations that fall into each upper_bound bucket for time spent in request queue before processing starts (seconds)	count	-
`litellm.request.queue_time_seconds.count`Number of observations for time spent in request queue before processing starts (seconds) in the time period	count	-
`litellm.request.queue_time_seconds.sum`Time spent in request queue before processing starts (seconds)	count	second
`litellm.request.total_latency.metric.bucket`Number of observations that fall into each upper_bound total latency bucket (seconds) for a request to LiteLLM	count	-
`litellm.request.total_latency.metric.count`Number of total latency observations (seconds) for a request to LiteLLM in the time period	count	-
`litellm.request.total_latency.metric.sum`Total latency (seconds) for a request to LiteLLM	count	second
`litellm.requests.metric.count`Deprecated - use litellm.proxy.total_requests.metric.count. Number of LLM calls to litellm in the time period - track total per API Key, team, user	count	request
`litellm.reset_budget_job.failed_requests.count`Number of failed requests for reset_budget_job service in the time period	count	error
`litellm.reset_budget_job.latency.bucket`Latency for reset_budget_job service	count	-
`litellm.reset_budget_job.total_requests.count`Number of requests for reset_budget_job service in the time period	count	request
`litellm.router.failed_requests.count`Number of failed requests for router service in the time period	count	error
`litellm.router.latency.bucket`Number of observations that fall into each upper_bound latency bucket for router service	count	-
`litellm.router.latency.count`Number of latency observations for router service in the time period	count	-
`litellm.router.latency.sum`Latency for router service	count	millisecond
`litellm.router.total_requests.count`Number of requests for router service in the time period	count	request
`litellm.self.failed_requests.count`Number of failed requests for self service in the time period	count	error
`litellm.self.latency.bucket`Number of observations that fall into each upper_bound latency bucket for self service	count	-
`litellm.self.latency.count`Number of latency observations for self service in the time period	count	-
`litellm.self.latency.sum`Latency for self service	count	millisecond
`litellm.self.total_requests.count`Number of requests for self service in the time period	count	request
`litellm.spend.metric.count`Spend on LLM requests in the time period	count	-
`litellm.team.budget.remaining_hours.metric`Remaining hours for team budget to be reset	gauge	hour
`litellm.team.max_budget.metric`Maximum budget set for team	gauge	-
`litellm.team.members.metric`Number of members in a team	gauge	-
`litellm.teams_count`Total number of teams in LiteLLM	gauge	-
`litellm.total.tokens.count`Number of input + output tokens from LLM requests in the time period	count	token
`litellm.total_users`Total number of users in LiteLLM	gauge	user
`litellm.user.budget.remaining_hours.metric`Remaining hours for user budget to be reset	gauge	hour
`litellm.user.max_budget.metric`Maximum budget set for user	gauge	-
`litellm.video_duration_seconds.metric.count`Seconds of video generated, from usage.duration_seconds on video generation calls	count	second

Official documentation

Official documentation for the LiteLLM integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Official summary

Metric types

Metrics

Official documentation

Related products

Other Metrics integrations