How do I capture prompts and responses with the OpenTelemetry Python SDK?

Rius captures prompts and responses from the OpenTelemetry Python SDK when your spans carry the gen_ai.input.messages and gen_ai.output.messages attributes. The plain SDK does not set them for you, so you add them to each model-call span as JSON in the GenAI message shape. Once they arrive, the prompt and completion show on the span’s Input / Output tab in the Rius console.

How it works

The OpenTelemetry Python SDK and its OTLP exporter only send the attributes your code or an instrumentor puts on a span. Setting the endpoint and headers does not add prompt or response text.

Rius reads content from these attributes:

• gen_ai.input.messages and gen_ai.output.messages on the span for the model call. Each holds a JSON array of messages shaped as {"role", "parts": [...]}, with text parts written as {"type": "text", "content": ...}.
• gen_ai.system_instructions for a system prompt you send separately from the chat history, as a JSON array of text parts.
• input.value and output.value for generic input and output on any span, as JSON or plain text.

Messages must follow the GenAI shape for the console to render them. A span without content attributes shows “Content not captured” on the Input / Output tab. Its timings, tokens and cost are unaffected.

The plain SDK has no Rius privacy controls, such as the RIUS_CAPTURE_CONTENT setting that strips content at export in the Rius SDK. What you put in these attributes is what leaves your process, so redact before you set them. Instrumentors that already emit prompt and response attributes, such as OpenInference, need no manual step. Rius normalizes their attributes onto the same keys at ingest.

Set it up

1. Install opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http, and set the standard exporter variables. Vanilla OpenTelemetry SDKs covers this setup in full. The attribute limit matters once spans carry many attributes.

export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.eu.console.rius-glassflow.com"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer <your API key>"
export OTEL_SERVICE_NAME="my-agent"
export OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT=4096

2. Register a tracer provider with the http/protobuf exporter, then create a span for each model call and set the model, the tokens and the messages. Replace call_the_model and the message text with your own. Serialize the messages with json.dumps in the GenAI shape.

import json
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
 
# With no arguments, the exporter reads the OTEL_EXPORTER_OTLP_* variables
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
 
tracer = trace.get_tracer("my-agent")
 
with tracer.start_as_current_span("chat gpt-4o") as span:
    span.set_attributes({
        "openinference.span.kind": "LLM",
        "gen_ai.operation.name": "chat",
        "gen_ai.request.model": "gpt-4o",
        "gen_ai.provider.name": "openai",
        "gen_ai.input.messages": json.dumps([
            {"role": "user", "parts": [{"type": "text", "content": "What is 2+2?"}]},
        ]),
    })
    response = call_the_model()
    span.set_attributes({
        "gen_ai.usage.input_tokens": response.usage.prompt_tokens,
        "gen_ai.usage.output_tokens": response.usage.completion_tokens,
        "gen_ai.response.model": response.model,
        "gen_ai.output.messages": json.dumps([
            {"role": "assistant", "parts": [{"type": "text", "content": response.text}]},
        ]),
    })

3. If you send a system prompt apart from the chat history, set it on the same span as gen_ai.system_instructions. For tool calls inside the conversation, use the tool_call and tool_call_response part types from the span attribute reference.

4. Do not send a cost attribute. Rius computes cost from the request model and the token counts.

5. Run one call, open Traces in the console, and open the new trace. The span’s Input / Output tab shows the prompt and the completion. The Trace LLM calls with OpenTelemetry guide covers the other ways to send traces.

What you get

• The prompt and the completion rendered on each model-call span’s Input / Output tab.
• Model name, token counts and a list-price cost estimate in US dollars on the same span, computed by Rius from the model and tokens you set.
• Your own control over content, since only the attributes you set are sent.
• The same result from any language, because Rius reads the attributes, not the SDK that wrote them.

When something else fits

• If you would rather not hand-build message JSON, the Rius SDK or an instrumentor such as OpenInference emits the message attributes for you, and the Rius SDK adds export-time privacy controls.
• If prompts hold sensitive text and you want no content stored, leave the content attributes off and send only the model and token attributes. Rius still computes cost and shows timings.

Related