Syntax
newrelic.agent.record_llm_feedback_event(trace_id, rating, category=None, message=None, metadata=None)Records custom feedback events for AI Large Language Model applications.
Requirements
Python agent version 9.8.0 or higher.
Description
This API records a feedback event LlmFeedbackMessage that can be viewed and queried in the New Relic UI. Feedback events correlate trace IDs between an AI-generated message and the feedback an end user submitted about it. To correlate messages with feedback, you can obtain the trace ID of the active transaction via a call to current_trace_id right after the call that generates the AI message. Pass the trace ID to the feedback call later when a user provides feedback.
In many cases, the endpoint for AI messages are recorded in different places from the feedback endpoint. They may happen in different transactions. It's important to:
- Make sure that the trace ID is captured inside the endpoint that generates the AI message.
- Pass that trace ID inside the endpoint that records the feedback.
Parameters
Parameter | Description |
|---|---|
string | Required. ID of the trace where the chat completion(s) related to the feedback occurred. This ID can be obtained via a call to |
string or int | Required. Rating provided by an end user (ex: “Good/Bad”, “1-10”). |
string | Optional. Category of the feedback provided by the end user (ex: “informative”, “inaccurate”). |
string | Optional. Freeform text feedback from an end user. |
dict | Optional. Set of key-value pairs to store any other desired data to submit with the feedback event. |
Return values
None.
Examples
Obtain trace ID and record feedback
Example of recording a feedback event:
import newrelic.agent
def get_message(request): trace_id = newrelic.agent.current_trace_id()
def post_feedback(request): newrelic.agent.record_llm_feedback_event(trace_id=request.trace_id, rating=request.rating, metadata= {"my_key": "my_val"})