obsdog

BentoML integration metrics

The official Datadog integration for monitoring BentoML. It collects 11 metrics.

Official summary

BentoML is an open-source framework for ML model deployment. This integration collects BentoML service metrics.

Metric types

  • count (occurrences in the interval): 7
  • gauge (value at a point in time): 4

Metrics

Descriptions are the official definitions, unedited.

MetricTypeUnit
bentoml.endpoint_livezA liveness probe endpoint that checks if the Service is still alive and needs to restart. This metrics reports a 1 if the endpoint is healthy, 0 otherwise.gauge-
bentoml.endpoint_readyzA readiness probe endpoint that indicates if the Service is ready to accept traffic. This metrics reports a 1 if the endpoint is healthy, 0 otherwise.gauge-
bentoml.service.adaptive_batch_size.bucketThe number of observations since last data collection that fall within a specific upper_bound tag for adaptive batch sizes used during Service execution histogram measurements.count-
bentoml.service.adaptive_batch_size.countThe number of observations since last data collection for adaptive batch sizes used during Service execution histogram measurements.count-
bentoml.service.adaptive_batch_size.sumThe total batch size units across all observations since last data collection for adaptive batch sizes used during Service execution histogram measurements.count-
bentoml.service.request.countThe number of new requests that a Service has processed since last submissioncountrequest
bentoml.service.request.duration.bucketThe number of observations since last data collection that fall within a specific upper_bound tag for request processing duration histogram measurements.count-
bentoml.service.request.duration.countThe number of requests processed since last data collection histogram measurements.count-
bentoml.service.request.duration.sumThe total sum of request processing time in seconds across all observations since last data collection histogram measurements.countsecond
bentoml.service.request.in_progressThe number of requests that are currently being processed by a Service.gaugerequest
bentoml.service.time_since_last_requestThe amount of time in seconds since the last request was processed by a Service.gaugesecond

Official documentation

Official documentation for the BentoML integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Related products

IntegrationsIntegrations are the bundle of entry points that pour data from external systems such as AWS and MySQL into Datadog. Where you configure them splits into two families, the Agent side and the cloud account side, and which one you choose changes both what you can see and how billing is counted.Foundation and data collectionLLM ObservabilityLLM Observability records a single request to an application with an embedded LLM as a trace, tracks the inputs and outputs, tokens, latency, and errors of each model call, and turns response quality into numbers through evaluations. The only thing billed is the LLM span, which represents a model call.APM and tracingLog ManagementLog Management is the product that gathers logs from servers, containers, and cloud services in one place and connects them to search, retention, and notification. It has a distinctive structure in which ingestion and indexing are billed separately.Log managementDatadog AgentSoftware that runs resident on the servers and containers you monitor and delivers metrics, logs, and traces to Datadog.Foundation and data collection

Other AI/ML integrations

Databricks16 metricsNvidia DCGM Exporter45 metricsHugging Face TGI54 metricsKubeflow48 metricsLangChain6 metricsMicrosoft Copilot21 metricsMilvus386 metricsNvidia NIM34 metricsNvidia Triton42 metricsOpenAI29 metricsRay146 metricsSGLang99 metrics