Send your GenAI traces to New Relic with OpenTelemetry

Once your application starts sending OpenTelemetry (OTel) GenAI traces to New Relic, the AI Monitoring view shows:

Every LLM call made by your service — prompt, completion, model, latency, token counts, finish reason.
Tool calls and agent steps as part of the full request trace, so you can see why a model made a decision.
Errors, rate-limit failures, and slow calls surfaced alongside the rest of your APM data.

No New Relic agent is required. If your app already emits OpenTelemetry traces, you are most of the way there.

Before you start

You will need:

A New Relic ingest license key.
An application that calls an LLM. Python examples for OpenAI, Anthropic, LangChain, and AWS Bedrock are shown below. Other languages with OpenTelemetry GenAI instrumentation work the same way.
Network egress from your app to the New Relic OTLP endpoint for your region on port 443. For the full list of regional endpoints (US, EU, and JP), supported ports, and compression settings, see Send data to New Relic using OpenTelemetry Protocol (OTLP).

Important

OTel GenAI instrumentations can capture the exact prompts and completions your users send and receive. That may include PII, credentials, or regulated data. Content capture is off by default. Before turning it on, review your organization's data-handling policies and see Control what prompts and completions are sent.

Point your OpenTelemetry exporter at New Relic

Set these environment variables on the process running your application:

bash

$# Replace with your region's OTLP endpoint — see the prerequisites above for the full list
$export OTEL_EXPORTER_OTLP_ENDPOINT="<<YOUR_OTLP_ENDPOINT>>"
$export OTEL_EXPORTER_OTLP_HEADERS="api-key=$NEW_RELIC_LICENSE_KEY"
$
$# Name that will identify this service in New Relic
$export OTEL_SERVICE_NAME="my-genai-service"

Add the right instrumentation package

Pick the section that matches what your application uses. You can combine more than one — for example, a LangChain app that calls Bedrock would install both.

Conseil

Exact package names and versions evolve quickly in the OpenTelemetry GenAI ecosystem. Always check the OpenTelemetry Python contrib registry and the New Relic AI Monitoring docs for the package your team currently supports.

Install the official OpenAI GenAI instrumentation from the OpenTelemetry project, then either run your app with opentelemetry-instrument or call the instrumentor explicitly:

from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
OpenAIInstrumentor().instrument()

from openai import OpenAI
OpenAI().chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

Install an OpenTelemetry Anthropic instrumentation (available from the OpenTelemetry contrib registry and from community distributions such as OpenLLMetry). Then:

from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
AnthropicInstrumentor().instrument()

import anthropic
anthropic.Anthropic().messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)

LangChain instrumentation covers the chain, any tool calls, and the underlying model client in one package. Install the community LangChain instrumentation of your choice and enable it at startup:

from opentelemetry.instrumentation.langchain import LangchainInstrumentor
LangchainInstrumentor().instrument()

Your existing ChatOpenAI, ChatAnthropic, or similar chains will be traced automatically. Tool and agent steps appear as child spans under the parent chain invocation.

Bedrock traffic goes through the AWS SDK. Install the AWS SDK (botocore) OpenTelemetry instrumentation and enable it:

from opentelemetry.instrumentation.botocore import BotocoreInstrumentor
BotocoreInstrumentor().instrument()

import boto3, json
boto3.client("bedrock-runtime", region_name="us-east-1").invoke_model(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    body=json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 256,
        "messages": [{"role": "user", "content": "Hello"}],
    }),
)

Verify that your installed version emits GenAI attributes (gen_ai.system, gen_ai.request.model, and token usage) for Bedrock calls — support has landed incrementally across versions.

Verify that your data is arriving

Restart your application so the new environment variables and instrumentation take effect.
Trigger a small amount of real traffic — a handful of chat completions is enough.

In New Relic, open Query Your Data and run:

SELECT count(*) FROM Span
WHERE gen_ai.system IS NOT NULL OR gen_ai.provider.name IS NOT NULL
SINCE 30 minutes ago
FACET service.name, gen_ai.operation.name

Within a few minutes (2-5 min) you should see rows faceted by your service name and operation (e.g. chat, completion, embeddings).
Go to APM & Services > AI Monitoring and select your service to see the AI Traces view populate.

Control what prompts and completions are sent

By default, most OpenTelemetry GenAI instrumentations do not capture the text of prompts or completions. Spans will include the model, token counts, latency, and status, but not the message bodies.

To make message content visible in AI Monitoring, opt in explicitly. The flag name depends on the instrumentation package you installed — check the documentation for that specific package. Common examples include:

OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true for OpenTelemetry-maintained instrumentations.
TRACELOOP_TRACE_CONTENT=true for community (OpenLLMetry-family) instrumentations.

Before enabling content capture in production:

Review the data your prompts contain. If your users submit PII, credentials, or regulated data, we recommend using drop filters or attribute-level obfuscation to strip or hash specific fields before they are sent to New Relic. See New Relic's data-handling documentation.
Check your organization's retention and compliance requirements (GDPR, HIPAA, PCI, etc.).

If you only need latency, token, and error visibility — not message bodies — leave content capture disabled.

Troubleshoot

What you see	What to try
The query in step 3 returns zero rows	Confirm your OTLP endpoint matches your account's region, and that the `api-key` header holds a valid ingest license key (not a user key).
Spans arrive but the AI Monitoring view is empty for your service	The spans may not be tagged with the expected GenAI attributes. Upgrade the instrumentation package, or contact New Relic support with a sample span.
Long prompts are cut off	Raise `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` and `OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT` on the application side.
Token counts are missing	The LLM provider returned them, but your instrumentation version does not record them yet. Upgrade the instrumentation package.
AI Monitoring shows traces but prompts/completions are blank	Content capture is disabled. See Control what prompts and completions are sent.
You expect AI Monitoring to be available but the view does not appear	Contact your New Relic account team to confirm AI Monitoring is enabled on your account.

Get help

If you are stuck after working through the steps above:

File a New Relic support case and include a sample span payload and your service name.
Reach out to your New Relic account team for account-level configuration questions.
For general OpenTelemetry GenAI questions, the OpenTelemetry GenAI semantic conventions are the authoritative reference.