slinerx.blogg.se - Kafka exporter

The earliest available offset is used in the calculation of other metrics provided, so it is exported for informational purposes. Kafka Lag Exporter will calculate a set of partitions for all consumer groups available and then poll for the earliest available offset. The earliest offset available for topic partition. For example, the accompanying Grafana dashboard makes use of it to visualize the last produced offset and the last consumed offset in certain panels.

The last produced offset is used in the calculation of other metrics provided, so it is exported for informational purposes.

Kafka Lag Exporter will calculate a set of partitions for all consumer groups available and then poll for the last produced offset. The latest offset available for topic partition. The sum of the difference between the last produced offset and the last consumed offset of all partitions in this topic for this group. The sum of the difference between the last produced offset and the last consumed offset of all partitions for this group. The highest (maximum) lag in time for a given consumer group. Kafka_consumergroup_group_max_lag_seconds The highest (maximum) lag in offsets for a given consumer group. Labels: cluster_name, group, is_simple_consumer For more information on how this is calculated read the Estimate consumer group lag in time section below. This metric correlates with lag in offsets. The difference between the last produced offset and the last consumed offset for this partition in this topic partition for this group. The offset of the last consumed offset for this partition in this topic partition for this group. Labels: cluster_name, group, topic, partition, member_host, consumer_id, client_id Helm and when enabling the Kubernetes pod self-discovery features within Prometheus server, Prometheus server willĪutomatically detect the HTTP endpoint and scrape its data. Kafka LagĮxporter exposes several metrics as an HTTP endpoint that can be readily scraped by Prometheus. Prometheus is a standard way to represent metrics in a modern cross-platform manner.

Filtering Metrics without Prometheus Server.

Monitor Kafka Consumer Group Latency with Kafka Lag Exporter. Kafka Lag Exporter interpolates latency based on observed latest committed offset measurements of consumer groups.įor more information about Kafka Lag Exporter's features see Lightbend's blog post: If you like using this project and would like to support its development, please consider a donation using Patreon. Kafka Lag Exporter is maintained by Sean Glover ( and a community of contributors. Kafka Lag Exporter is an Akka Typed application written in Scala. It can run anywhere, but it provides features to run easily on Kubernetes clusters against Strimzi Kafka clusters using the Prometheus and Grafana monitoring stack. Kafka Lag Exporter makes it easy to view the offset lag and calculate an estimate of latency (residence time) of your Apache Kafka consumer groups. Monitor Kafka Consumer Group Latency with Kafka Lag Exporter Overview