---
title: Find and query Kafka data in New Relic
source: https://docs.newrelic.com/docs/opentelemetry/integrations/kafka/find-and-query-data
---

Once your Kafka monitoring is configured with OpenTelemetry, your metrics will appear across multiple locations in New Relic. This guide shows you where to find your data and how to query it effectively.

## Navigate to your Kafka data [#find-data]

Your Kafka metrics appear in several places across the New Relic platform, each optimized for different use cases:

### 1. Entity explorer

Best for: Quick health checks and entity relationships

1.  Go to **[one.newrelic.com](https://one.newrelic.com) > All capabilities > On host integrations**
2.  View Kafka broker, cluster, and topics entity types

![Entity explorer showing Kafka broker, cluster, and topics entity types](https://docs.newrelic.com/images/otel-kafka-entityexplorer.webp "OTel Kafka Entity Explorer")

### 2. Queues & Streams

Best for: Message queue focused monitoring

1.  Go to **[one.newrelic.com](https://one.newrelic.com) > All capabilities > Queues & Streams**
2.  Filter by `provider=opentelemetry` and click the account row where you instrumented Kafka to view your data

![Queues & Streams view showing Kafka data filtered by OpenTelemetry provider](https://docs.newrelic.com/images/otel-kafka-queues-streams.webp "OTel Kafka Queues & Streams")

/\* ### 3. Third-party services
Best for: Integration-focused monitoring

1\. Go to \*\*\[one.newrelic.com](https://one.newrelic.com) > All capabilities > Infrastructure > Third-party services\*\*
2\. Select \*\*Kafka (OpenTelemetry)\*\* > \*\*OpenTelemetry Kafka\*\*

 \*/

### 3. Pre-built dashboards

Best for: Comprehensive monitoring and visualization

1.  Go to **[one.newrelic.com](https://one.newrelic.com) > Dashboards > Recommended dashboards (View all)**
2.  Search for **"OpenTelemetry Kafka"**

![Pre-built dashboard showing Kafka cluster health, network throughput, and broker metrics](https://docs.newrelic.com/images/otel-kafka-prebuilt-dashboards.webp "OpenTelemetry Kafka")

## Query Kafka data with NRQL [#query-data]

Use [NRQL](https://docs.newrelic.com/docs/nrql/get-started/introduction-nrql-new-relics-query-language) to create custom queries and dashboards for your Kafka data.

**Basic queries**

**View all Kafka metrics:**

````sql
FROM Metric SELECT *
WHERE instrumentation.provider='opentelemetry'
AND kafka.cluster.name IS NOT NULL
LIMIT 100
```

**Monitor specific cluster:**
```sql
FROM Metric SELECT *
WHERE kafka.cluster.name = 'production-cluster'
AND instrumentation.provider='opentelemetry'
```

````

**Consumer lag monitoring**

**Consumer lag by group:**

````sql
FROM Metric SELECT latest(kafka.consumer_group.lag)
WHERE instrumentation.provider='opentelemetry'
FACET group, topic
SINCE 1 hour ago TIMESERIES
```

**High consumer lag alert:**
```sql
FROM Metric SELECT max(kafka.consumer_group.lag)
WHERE instrumentation.provider='opentelemetry'
AND group = 'your-consumer-group'
```

````

**Broker health queries**

**Active brokers count:**

````sql
FROM Metric SELECT latest(kafka.brokers)
WHERE instrumentation.provider='opentelemetry'
AND kafka.cluster.name = 'your-cluster'
TIMESERIES
```

**Under-replicated partitions:**
```sql
FROM Metric SELECT latest(kafka.partition.under_replicated)
WHERE instrumentation.provider='opentelemetry'
FACET broker.id SINCE 1 hour ago
```

````

**Performance monitoring**

**Message throughput by topic:**

````sql
FROM Metric SELECT rate(sum(kafka.prod.msg.count), 1 minute) as 'Messages/min'
WHERE instrumentation.provider='opentelemetry'
FACET topic SINCE 1 hour ago TIMESERIES
```

**Network I/O by broker:**
```sql
FROM Metric SELECT rate(sum(kafka.network.io), 1 minute) as 'Bytes/min'
WHERE instrumentation.provider='opentelemetry'
FACET broker.id, direction SINCE 1 hour ago TIMESERIES
```

````

## Advanced queries [#advanced-queries]

For more comprehensive query examples organized by monitoring focus area:

**Cluster overview**

Get high-level cluster statistics and health:

````sql
FROM Metric SELECT
  max(kafka.brokers) AS 'Active Brokers',
  max(kafka.cluster.topic.count) AS 'Total Topics',
  max(kafka.cluster.partition.count) AS 'Total Partitions',
  latest(kafka.partition.offline) AS 'Offline Partitions'
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
TIMESERIES
```

````

**Broker health**

Monitor individual broker health and leadership:

````sql
FROM Metric SELECT
  latest(kafka.partition.under_replicated) AS 'Under-replicated Partitions',
  latest(kafka.broker.leader.count) AS 'Leader Count',
  latest(kafka.partition.count) AS 'Total Partitions'
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET broker.id
TIMESERIES
```

````

**Topic throughput and I/O**

Monitor message rates and data throughput by topic:

````sql
FROM Metric SELECT
  rate(sum(kafka.prod.msg.count), 1 minute) AS 'Messages/min',
  rate(filter(sum(kafka.topic.io), WHERE direction = 'in'), 1 minute) AS 'Bytes In/min',
  rate(filter(sum(kafka.topic.io), WHERE direction = 'out'), 1 minute) AS 'Bytes Out/min'
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET topic
TIMESERIES
```

````

**Consumer lag**

Track consumer group lag and processing delays:

````sql
FROM Metric SELECT
  latest(kafka.consumer_group.lag_sum) AS 'Total Lag'
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET group
TIMESERIES
```

````

**Request latency**

Monitor request processing times and performance:

````sql
FROM Metric SELECT
  latest(kafka.request.time.avg) AS 'Avg Latency',
  latest(`kafka.request.time.99p`) AS '99th Percentile'
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET type
TIMESERIES
```

````

**JVM health**

Monitor JVM metrics for Kafka brokers:

````sql
FROM Metric SELECT
  (latest(jvm.memory.heap.used) / latest(jvm.memory.heap.max)) * 100 AS 'Heap Usage %',
  rate(sum(jvm.gc.collections.count), 1 minute) AS 'GC Count/min'
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET broker.id
TIMESERIES
```

````

## Create custom dashboards [#custom-dashboards]

Combine multiple queries to create comprehensive Kafka monitoring dashboards:

1.  Go to [one.newrelic.com](https://one.newrelic.com) > **Dashboards**
2.  Click **Create a dashboard**
3.  Add widgets using the NRQL queries above
4.  Organize widgets by focus area:
    -   Cluster health: Broker count, under-replicated partitions, controller status
    -   Consumer performance: Lag, throughput, group membership
    -   Topic metrics: Message rates, partition counts, replication status
    -   Resource utilization: Network I/O, JVM metrics, disk usage

## Set up alerts [#alerts]

> #### 💡 TIP
>
> New Relic provides pre-configured golden metric alerts for Kafka. When creating a new alert condition, select **Guided mode** → **Host integrations** → **Kafka** (brokers, cluster, or topics) to view recommended alert templates.

You can also create custom alerts using NRQL queries:

**High consumer lag**

Alert when consumer lag exceeds your acceptable threshold:

````sql
FROM Metric SELECT
  latest(kafka.consumer_group.lag_sum)
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET group
```

**Recommended threshold**: Alert when lag > 10,000 messages for critical consumer groups.

````

**Under-replicated partitions**

Alert when partitions lose replication:

````sql
FROM Metric SELECT
  sum(kafka.partition.under_replicated)
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
```

**Recommended threshold**: Alert when any partitions become under-replicated (> 0).

````

**Offline partitions**

Alert immediately when partitions go offline:

````sql
FROM Metric SELECT
  latest(kafka.partition.offline)
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
```

**Recommended threshold**: Alert immediately when any partitions go offline (> 0).

````

**Partitions under minimum ISR**

Alert when partitions fall below minimum in-sync replicas:

````sql
FROM Metric SELECT
  latest(kafka.partition.under_min_isr)
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET broker.id
```

**Recommended threshold**: Alert when any partitions fall below minimum ISR (> 0).

````

**High JVM memory usage**

Alert when heap usage is critical:

````sql
FROM Metric SELECT
  (latest(jvm.memory.heap.used) / latest(jvm.memory.heap.max)) * 100
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET broker.id
```

**Recommended threshold**: Alert when heap usage > 85% for sustained periods.

````

**Request failures**

Alert on high request failure rates:

````sql
FROM Metric SELECT
  rate(sum(kafka.request.failed), 1 minute)
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
FACET type
```

**Recommended threshold**: Alert when failure rate > 1% of total requests.

````

**Unclean leader elections**

Alert on unclean leader elections (indicates broker failures):

````sql
FROM Metric SELECT
  rate(sum(kafka.unclean.election.rate), 1 minute)
WHERE kafka.cluster.name = 'production-cluster'
  AND instrumentation.provider='opentelemetry'
```

**Recommended threshold**: Alert on any unclean leader elections (> 0).

````

## Next steps [#next-steps]

-   [Kafka metrics reference](https://docs.newrelic.com/docs/opentelemetry/integrations/kafka/metrics-reference) - Complete list of available metrics with descriptions
-   [Create NRQL alert conditions](https://docs.newrelic.com/docs/alerts/create-alert/create-alert-condition/create-nrql-alert-conditions/) - Set up proactive monitoring
-   [NRQL tutorial](https://docs.newrelic.com/docs/query-your-data/nrql-new-relic-query-language/get-started/introduction-nrql-new-relics-query-language) - Learn advanced querying techniques