obsdog

AWS Inferentia and AWS Trainium Monitoring integration metrics

The official Datadog integration for monitoring AWS Inferentia and AWS Trainium Monitoring. It collects 33 metrics.

Official summary

Monitor the performance and usage of AWS Inferentia/Trainium instances and the Neuron SDK.

Metric types

  • gauge (value at a point in time): 26
  • count (occurrences in the interval): 7

Metrics

Descriptions are the official definitions, unedited.

MetricTypeUnit
aws_neuron.execution.errors.countExecution errors totalcount-
aws_neuron.execution.errors_createdExecution errors totalgauge-
aws_neuron.execution.latency_secondsExecution latency in secondsgaugesecond
aws_neuron.execution.status.countExecution status totalcount-
aws_neuron.execution.status_createdExecution status totalgauge-
aws_neuron.hardware_ecc_events.countHardware ecc events totalcount-
aws_neuron.hardware_ecc_events_createdHardware ecc events totalgauge-
aws_neuron.instance_infoEC2 instance informationgauge-
aws_neuron.neuron_hardware_infoNeuron Hardware Informationgauge-
aws_neuron.neuron_runtime.memory_used_bytesRuntime memory used bytesgaugebyte
aws_neuron.neuron_runtime.vcpu_usage_ratioRuntime vCPU utilization ratiogaugefraction
aws_neuron.neuroncore.memory_usage.constantsNeuronCore memory utilization for constantsgaugebyte
aws_neuron.neuroncore.memory_usage.model.codeNeuronCore memory utilization for model_codegaugebyte
aws_neuron.neuroncore.memory_usage.model.shared_scratchpadNeuronCore memory utilization for model_shared_scratchpadgaugebyte
aws_neuron.neuroncore.memory_usage.runtime_memoryNeuronCore memory utilization for runtime_memorygaugebyte
aws_neuron.neuroncore.memory_usage.tensorsNeuronCore memory utilization for tensorsgaugebyte
aws_neuron.neuroncore.utilization_ratioNeuronCore utilization ratiogaugefraction
aws_neuron.process.cpu_seconds.countTotal user and system CPU time spent in seconds.countsecond
aws_neuron.process.max_fdsMaximum number of open file descriptors.gauge-
aws_neuron.process.open_fdsNumber of open file descriptors.gauge-
aws_neuron.process.resident_memory_bytesResident memory size in bytes.gaugebyte
aws_neuron.process.start_time_secondsStart time of the process since unix epoch in seconds.gaugesecond
aws_neuron.process.virtual_memory_bytesVirtual memory size in bytes.gaugebyte
aws_neuron.python_gc.collections.countNumber of times this generation was collectedcount-
aws_neuron.python_gc.objects_collected.countObjects collected during gccount-
aws_neuron.python_gc.objects_uncollectable.countUncollectable objects found during GCcount-
aws_neuron.python_infoPython platform informationgauge-
aws_neuron.system.memory.total_bytesSystem memory total_bytes bytesgaugebyte
aws_neuron.system.memory.used_bytesSystem memory used_bytes bytesgaugebyte
aws_neuron.system.swap.total_bytesSystem swap total_bytes bytesgaugebyte
aws_neuron.system.swap.used_bytesSystem swap used_bytes bytesgaugebyte
aws_neuron.system.vcpu.countSystem vCPU countgauge-
aws_neuron.system.vcpu.usage_ratioSystem CPU utilization ratiogaugefraction

Official documentation

Official documentation for the AWS Inferentia and AWS Trainium Monitoring integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Related products

IntegrationsIntegrations are the bundle of entry points that pour data from external systems such as AWS and MySQL into Datadog. Where you configure them splits into two families, the Agent side and the cloud account side, and which one you choose changes both what you can see and how billing is counted.Foundation and data collectionLLM ObservabilityLLM Observability records a single request to an application with an embedded LLM as a trace, tracks the inputs and outputs, tokens, latency, and errors of each model call, and turns response quality into numbers through evaluations. The only thing billed is the LLM span, which represents a model call.APM and tracingInfrastructure MonitoringInfrastructure Monitoring is the product that visualizes the state of servers, containers, and cloud resources at the host level. The host map and process list let you grasp at a glance which host is struggling right now.Infrastructure monitoringLog ManagementLog Management is the product that gathers logs from servers, containers, and cloud services in one place and connects them to search, retention, and notification. It has a distinctive structure in which ingestion and indexing are billed separately.Log management

Other AWS integrations

Amazon ECS on AWS Fargate34 metricsAmazon EKS on AWS Fargate3 metricsAmazon MSK (Agent)421 metrics