StreamNative integration metrics

The official Datadog integration for monitoring StreamNative. It collects 125 metrics.

Definitions as of 2024-10-01

Official summary

Gain insights into StreamNative metrics data.

Metrics

Metric types

gauge (value at a point in time): 125

Metrics

Descriptions are the official definitions, unedited.

Metric	Type	Unit
`streamnative.health.pulsar_detector_e2e_latency_ms`The latency distribution from message sending to message consumption	gauge	millisecond
`streamnative.health.pulsar_detector_geo_latency_ms`The latency distribution Latency distribution from message sending to message consumption across clusters	gauge	millisecond
`streamnative.health.pulsar_detector_publish_latency_ms`The latency distribution of message sending	gauge	millisecond
`streamnative.health.pulsar_detector_pulsar_sla_messaging_up`The gauge for indicating the messaging service up or down	gauge	-
`streamnative.health.pulsar_detector_pulsar_sla_webservice_up`The gauge for indicating the webservice up or down	gauge	-
`streamnative.kafka_connect.jvm_gc_collection_seconds_sum`Time spent in a given JVM garbage collector in seconds.	gauge	second
`streamnative.kafka_connect.jvm_memory_committed_bytes`Committed bytes of a given JVM memory area	gauge	byte
`streamnative.kafka_connect.jvm_memory_init_bytes`Initial bytes of a given JVM memory area	gauge	byte
`streamnative.kafka_connect.jvm_memory_max_bytes`Max bytes of a given JVM memory area	gauge	byte
`streamnative.kafka_connect.jvm_memory_used_bytes`Used bytes of a given JVM memory area	gauge	byte
`streamnative.kafka_connect.kafka_connect_connector_task_batch_size_avg`The average size of the batches processed by the connector	gauge	-
`streamnative.kafka_connect.kafka_connect_connector_task_batch_size_max`The maximum size of the batches processed by the connector	gauge	-
`streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_avg_time_ms`The average time in milliseconds taken by this task to commit offsets	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_failure_percentage`The average percentage of this task's offset commit attempts that failed	gauge	fraction
`streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_max_time_ms`The maximum time in milliseconds taken by this task to commit offsets	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_success_percentage`The average percentage of this task's offset commit attempts that succeeded	gauge	fraction
`streamnative.kafka_connect.kafka_connect_connector_task_pause_ratio`The fraction of time this task has spent in the pause state	gauge	fraction
`streamnative.kafka_connect.kafka_connect_connector_task_running_ratio`The fraction of time this task has spent in the running state	gauge	fraction
`streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_completion`The total number of offset commit completions that were completed successfully	gauge	offset
`streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_completion_rate`The average per-second number of offset commit completions that were completed successfully	gauge	offset/second
`streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_seq_no`The current sequence number for offset commits	gauge	-
`streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_skip`The total number of offset commit completions that were received too late and skipped/ignored	gauge	offset
`streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_skip_rate`The average per-second number of offset commit completions that were received too late and skipped/ignored	gauge	offset/second
`streamnative.kafka_connect.kafka_connect_sink_task_partition_count`The number of topic partitions assigned to this task belonging to the named sink connector in this worker	gauge	-
`streamnative.kafka_connect.kafka_connect_sink_task_put_batch_avg_time_ms`The average time taken by this task to put a batch of sinks records	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_sink_task_put_batch_max_time_ms`The maximum time taken by this task to put a batch of sinks records	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_active_count`The number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task	gauge	record
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_active_count_avg`The average number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task	gauge	record
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_active_count_max`The maximum number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task	gauge	record
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_read`The total number of records read from Kafka by this task belonging to the named sink connector in this worker, since the task was last restarted	gauge	record
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_read_rate`The average per-second number of records read from Kafka for this task belonging to the named sink connector in this worker. This is before transformations are applied	gauge	record/second
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_send`The total number of records output from the transformations and sent/put to this task belonging to the named sink connector in this worker, since the task was last restarted	gauge	record
`streamnative.kafka_connect.kafka_connect_sink_task_sink_record_send_rate`The average per-second number of records output from the transformations and sent/put to this task belonging to the named sink connector in this worker	gauge	record/second
`streamnative.kafka_connect.kafka_connect_source_task_poll_batch_avg_time_ms`The average time in milliseconds taken by this task to poll for a batch of source records	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_source_task_poll_batch_max_time_ms`The maximum time in milliseconds taken by this task to poll for a batch of source records	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_source_task_source_record_active_count`The number of records that have been produced by this task but not yet completely written to Kafka	gauge	record
`streamnative.kafka_connect.kafka_connect_source_task_source_record_active_count_avg`The average number of records that have been produced by this task but not yet completely written to Kafka	gauge	record
`streamnative.kafka_connect.kafka_connect_source_task_source_record_active_count_max`The maximum number of records that have been produced by this task but not yet completely written to Kafka	gauge	record
`streamnative.kafka_connect.kafka_connect_source_task_source_record_poll`The total number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker	gauge	record
`streamnative.kafka_connect.kafka_connect_source_task_source_record_poll_rate`The average per-second number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker	gauge	record/second
`streamnative.kafka_connect.kafka_connect_source_task_source_record_write`The number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker, since the task was last restarted	gauge	record
`streamnative.kafka_connect.kafka_connect_source_task_source_record_write_rate`The average per-second number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker	gauge	record/second
`streamnative.kafka_connect.kafka_connect_task_error_deadletterqueue_produce_failures`The number of failed writes to the dead letter queue	gauge	-
`streamnative.kafka_connect.kafka_connect_task_error_deadletterqueue_produce_requests`The number of attempted writes to the dead letter queue	gauge	-
`streamnative.kafka_connect.kafka_connect_task_error_last_error_timestamp`The epoch timestamp when this task last encountered an error	gauge	millisecond
`streamnative.kafka_connect.kafka_connect_task_error_total_errors_logged`The total number of errors that were logged	gauge	error
`streamnative.kafka_connect.kafka_connect_task_error_total_record_errors`The total number of record processing errors in this task	gauge	record
`streamnative.kafka_connect.kafka_connect_task_error_total_record_failures`The total number of record processing failures in this task	gauge	record
`streamnative.kafka_connect.kafka_connect_task_error_total_records_skipped`The total number of records skipped due to errors	gauge	record
`streamnative.kafka_connect.kafka_connect_task_error_total_retries`The total number of operations retried	gauge	operation
`streamnative.kafka_connect.kafka_connect_worker_connector_destroyed_task_count`The number of destroyed tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.kafka_connect_worker_connector_failed_task_count`The number of failed tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.kafka_connect_worker_connector_paused_task_count`The number of paused tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.kafka_connect_worker_connector_restarting_task_count`The number of restarting tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.kafka_connect_worker_connector_running_task_count`The number of running tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.kafka_connect_worker_connector_total_task_count`The number of tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.kafka_connect_worker_connector_unassigned_task_count`The number of unassigned tasks of the connector on the worker	gauge	task
`streamnative.kafka_connect.process_cpu_seconds_total`Total user and system CPU time spent in seconds	gauge	second
`streamnative.pulsar_resource.pulsar_consumers_count`The number of active consumers of the topic connected to this broker.	gauge	-
`streamnative.pulsar_resource.pulsar_entry_size_le_100_kb`The entry rate for a topic with entry size smaller than or equal to 100 kilobytes	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_128`The entry rate for a topic with entry size smaller than or equal to 128 bytes	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_16_kb`The entry rate for a topic with entry size smaller than or equal to 16 kilobytes	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_1_kb`The entry rate for a topic with entry size smaller than or equal to 1 kilobyte	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_1_mb`The entry rate for a topic with entry size smaller than or equal to 1 megabyte	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_2_kb`The entry rate for a topic with entry size smaller than or equal to 2 kilobytes	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_4_kb`The entry rate for a topic with entry size smaller than or equal to 4 kilobytes	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_512`The entry rate for a topic with entry size smaller than or equal to 512 bytes	gauge	fraction
`streamnative.pulsar_resource.pulsar_entry_size_le_overflow`The entry rate for a topic with entry size greater than 1 megabyte	gauge	fraction
`streamnative.pulsar_resource.pulsar_producers_count`The number of active producers of the topic connected to this broker	gauge	-
`streamnative.pulsar_resource.pulsar_rate_in`The total message rate of the namespace coming into this broker	gauge	message/second
`streamnative.pulsar_resource.pulsar_rate_out`The total message rate of the namespace going out from this broker	gauge	message/second
`streamnative.pulsar_resource.pulsar_storage_backlog_size`The total backlog size of the topics of this topic owned by this broker	gauge	byte
`streamnative.pulsar_resource.pulsar_storage_offloaded_size`The total amount of the data in this topic offloaded to the tiered storage	gauge	byte
`streamnative.pulsar_resource.pulsar_storage_read_rate`The total message batches (entries) read from the storage for this topic	gauge	-
`streamnative.pulsar_resource.pulsar_storage_size`The total storage size of the topics in this topic owned by this broker	gauge	byte
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_0_5`The entry rate for a topic where the storage write latency is less than or equal to 0.5 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_1`The entry rate for a topic where the storage write latency is less than or equal to 1 millisecond	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_10`The entry rate for a topic where the storage write latency is less than or equal to 10 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_100`The entry rate for a topic where the storage write latency is less than or equal to 100 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_1000`The entry rate for a topic where the storage write latency is less than or equal to 1000 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_20`The entry rate for a topic where the storage write latency is less than or equal to 20 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_200`The entry rate for a topic where the storage write latency is less than or equal to 200 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_5`The entry rate for a topic where the storage write latency is less than or equal to 5 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_50`The entry rate for a topic where the storage write latency is less than or equal to 50 milliseconds	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_latency_le_overflow`The entry rate for a topic where the storage write latency is greater than 1 second	gauge	fraction
`streamnative.pulsar_resource.pulsar_storage_write_rate`The total message batches (entries) written to the storage for this topic	gauge	-
`streamnative.pulsar_resource.pulsar_subscription_delayed`The total message batches (entries) are delayed for dispatching	gauge	-
`streamnative.pulsar_resource.pulsar_subscriptions_count`The number of Pulsar subscriptions of the topic served by this broker	gauge	-
`streamnative.pulsar_resource.pulsar_throughput_in`The total throughput of the topic coming into this broker	gauge	byte/second
`streamnative.pulsar_resource.pulsar_throughput_out`The total throughput of the topic going out from this broker	gauge	byte/second
`streamnative.pulsar_resource.pulsar_topics_count`The number of Pulsar topics of the namespace owned by this broker	gauge	-
`streamnative.sink_connector.jvm_gc_collection_seconds_sum`Time spent in a given JVM garbage collector in seconds	gauge	second
`streamnative.sink_connector.jvm_memory_bytes_committed`Committed bytes of a given JVM memory area	gauge	byte
`streamnative.sink_connector.jvm_memory_bytes_init`Initial bytes of a given JVM memory area	gauge	byte
`streamnative.sink_connector.jvm_memory_bytes_max`Max bytes of a given JVM memory area	gauge	byte
`streamnative.sink_connector.jvm_memory_direct_bytes_used`Used bytes of a given JVM memory area	gauge	byte
`streamnative.sink_connector.process_cpu_seconds_total`Total user and system CPU time spent in seconds	gauge	second
`streamnative.sink_connector.pulsar_sink_last_invocation`The timestamp of the last invocation of the sink	gauge	millisecond
`streamnative.sink_connector.pulsar_sink_received_1min_total`The total number of records received from sink in the last 1 minute	gauge	record
`streamnative.sink_connector.pulsar_sink_received_total`The total number of records received from sink	gauge	record
`streamnative.sink_connector.pulsar_sink_sink_exception`The exception from a sink	gauge	-
`streamnative.sink_connector.pulsar_sink_sink_exceptions_1min_total`The total number of sink exceptions in the last 1 minute	gauge	exception
`streamnative.sink_connector.pulsar_sink_sink_exceptions_total`The total number of sink exceptions	gauge	exception
`streamnative.sink_connector.pulsar_sink_system_exception`The exception from system code	gauge	-
`streamnative.sink_connector.pulsar_sink_system_exceptions_1min_total`The total number of system exceptions in the last 1 minute	gauge	exception
`streamnative.sink_connector.pulsar_sink_system_exceptions_total`The total number of system exceptions	gauge	exception
`streamnative.sink_connector.pulsar_sink_written_1min_total`The total number of records written to a Pulsar topic in the last 1 minute	gauge	record
`streamnative.sink_connector.pulsar_sink_written_total`The total number of records written to a Pulsar topic	gauge	record
`streamnative.source_connector.jvm_gc_collection_seconds_sum`Time spent in a given JVM garbage collector in seconds	gauge	second
`streamnative.source_connector.jvm_memory_bytes_committed`Committed bytes of a given JVM memory area	gauge	byte
`streamnative.source_connector.jvm_memory_bytes_init`Initial bytes of a given JVM memory area	gauge	byte
`streamnative.source_connector.jvm_memory_bytes_max`Max bytes of a given JVM memory area	gauge	byte
`streamnative.source_connector.jvm_memory_direct_bytes_used`Used bytes of a given JVM memory area	gauge	byte
`streamnative.source_connector.process_cpu_seconds_total`Total user and system CPU time spent in seconds	gauge	second
`streamnative.source_connector.pulsar_source_last_invocation`The timestamp of the last invocation of the source	gauge	millisecond
`streamnative.source_connector.pulsar_source_received_1min_total`The total number of records received from source in the last 1 minute	gauge	record
`streamnative.source_connector.pulsar_source_received_total`The total number of records received from source	gauge	record
`streamnative.source_connector.pulsar_source_source_exception`The exception from a source	gauge	-
`streamnative.source_connector.pulsar_source_source_exceptions_1min_total`The total number of source exceptions in the last 1 minute	gauge	exception
`streamnative.source_connector.pulsar_source_source_exceptions_total`The total number of source exceptions	gauge	exception
`streamnative.source_connector.pulsar_source_system_exception`The exception from system code	gauge	-
`streamnative.source_connector.pulsar_source_system_exceptions_1min_total`The total number of system exceptions in the last 1 minute	gauge	exception
`streamnative.source_connector.pulsar_source_system_exceptions_total`The total number of system exceptions	gauge	exception
`streamnative.source_connector.pulsar_source_written_1min_total`The total number of records written to a Pulsar topic in the last 1 minute	gauge	record
`streamnative.source_connector.pulsar_source_written_total`The total number of records written to a Pulsar topic	gauge	record

Official documentation

Official documentation for the StreamNative integration

Metric names and descriptions are the official Datadog integration definitions (Copyright Datadog, Inc., BSD-3-Clause), reproduced unedited. The full license terms are on the About page.

Official summary

Metric types

Metrics

Official documentation

Related products

Other Metrics integrations