Hugging Face TGI integration metrics

The official Datadog integration for monitoring Hugging Face TGI. It collects 54 metrics.

Definitions as of 2026-04-15

Official summary

Monitor the model serving performance and system health of your Hugging Face TGI servers.

Metric types

count (occurrences in the interval): 50
gauge (value at a point in time): 4

Metrics

Descriptions are the official definitions, unedited.

Metric	Type	Unit
`hugging_face_tgi.batch.concat.count`Number of batch concatenates	count	-
`hugging_face_tgi.batch.concat.duration.bucket`Batch concatenation duration distribution	count	-
`hugging_face_tgi.batch.concat.duration.count`Number of batch concatenation duration measurements	count	-
`hugging_face_tgi.batch.concat.duration.sum`Total batch concatenation duration	count	second
`hugging_face_tgi.batch.current.max_tokens`Maximum tokens the current batch will grow to	gauge	token
`hugging_face_tgi.batch.current.size`Current batch size	gauge	request
`hugging_face_tgi.batch.decode.duration.bucket`Batch decode duration distribution	count	-
`hugging_face_tgi.batch.decode.duration.count`Number of batch decode duration measurements	count	-
`hugging_face_tgi.batch.decode.duration.sum`Total batch decode duration	count	second
`hugging_face_tgi.batch.filter.duration.bucket`Batch filtering duration distribution	count	-
`hugging_face_tgi.batch.filter.duration.count`Number of batch filter duration measurements	count	-
`hugging_face_tgi.batch.filter.duration.sum`Total batch filter duration	count	second
`hugging_face_tgi.batch.forward.duration.bucket`Batch forward duration distribution	count	-
`hugging_face_tgi.batch.forward.duration.count`Number of batch forward duration measurements	count	-
`hugging_face_tgi.batch.forward.duration.sum`Total batch forward duration	count	second
`hugging_face_tgi.batch.inference.count`Total number of batch inferences	count	-
`hugging_face_tgi.batch.inference.duration.bucket`Batch inference duration distribution	count	-
`hugging_face_tgi.batch.inference.duration.count`Number of batch inference duration measurements	count	-
`hugging_face_tgi.batch.inference.duration.sum`Total batch inference duration	count	second
`hugging_face_tgi.batch.inference.success.count`Number of successful batch inferences	count	-
`hugging_face_tgi.batch.next.size.bucket`Next batch size distribution	count	-
`hugging_face_tgi.batch.next.size.count`Number of next batch size measurements	count	-
`hugging_face_tgi.batch.next.size.sum`Total next batch size	count	request
`hugging_face_tgi.queue.size`Number of requests waiting in the internal queue	gauge	request
`hugging_face_tgi.request.count`Total number of requests received	count	request
`hugging_face_tgi.request.duration.bucket`Request duration distribution	count	-
`hugging_face_tgi.request.duration.count`Number of request duration measurements	count	-
`hugging_face_tgi.request.duration.sum`Total request duration	count	second
`hugging_face_tgi.request.failure.count`Number of failed requests	count	request
`hugging_face_tgi.request.generated_tokens.bucket`Generated tokens per request distribution	count	-
`hugging_face_tgi.request.generated_tokens.count`Number of generated token measurements	count	-
`hugging_face_tgi.request.generated_tokens.sum`Total generated tokens	count	token
`hugging_face_tgi.request.inference.duration.bucket`Request inference duration distribution	count	-
`hugging_face_tgi.request.inference.duration.count`Number of request inference duration measurements	count	-
`hugging_face_tgi.request.inference.duration.sum`Total request inference duration	count	second
`hugging_face_tgi.request.input_length.bucket`Input token length per request distribution	count	-
`hugging_face_tgi.request.input_length.count`Number of input length measurements	count	-
`hugging_face_tgi.request.input_length.sum`Total input length	count	token
`hugging_face_tgi.request.max_new_tokens.bucket`Maximum new tokens per request distribution	count	-
`hugging_face_tgi.request.max_new_tokens.count`Number of max new tokens measurements	count	-
`hugging_face_tgi.request.max_new_tokens.sum`Total max new tokens	count	token
`hugging_face_tgi.request.mean_time_per_token.duration.bucket`Mean time per token duration distribution	count	-
`hugging_face_tgi.request.mean_time_per_token.duration.count`Number of mean time per token measurements	count	-
`hugging_face_tgi.request.mean_time_per_token.duration.sum`Total mean time per token duration	count	second
`hugging_face_tgi.request.queue.duration.bucket`Request queue duration distribution	count	-
`hugging_face_tgi.request.queue.duration.count`Number of request queue duration measurements	count	-
`hugging_face_tgi.request.queue.duration.sum`Total request queue duration	count	second
`hugging_face_tgi.request.skipped_tokens.count`Number of skipped token measurements	count	-
`hugging_face_tgi.request.skipped_tokens.quantile`Skipped tokens per request quantile	gauge	token
`hugging_face_tgi.request.skipped_tokens.sum`Total skipped tokens	count	token
`hugging_face_tgi.request.success.count`Number of successful requests	count	request
`hugging_face_tgi.request.validation.duration.bucket`Request validation duration distribution	count	-
`hugging_face_tgi.request.validation.duration.count`Number of request validation duration measurements	count	-
`hugging_face_tgi.request.validation.duration.sum`Total request validation duration	count	second

Official documentation

Official documentation for the Hugging Face TGI integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Official summary

Metric types

Metrics

Official documentation

Related products

Other AI/ML integrations