How do I capture prompts and responses with OpenInference instrumentors?

Rius captures prompts and responses from OpenInference instrumentors with no content setting, but you must raise the span attribute limit so the message attributes are not dropped. The instrumentor records the messages by default and Rius reads them. The wiring example in this article already raises the limit to 4096. If your setup builds its own tracer provider, add the limit yourself.

How it works

An OpenInference instrumentor writes the prompt and the reply as span attributes. It sets input.value and output.value for generic content. It also writes one attribute per message field, named llm.input_messages.* and llm.output_messages.*. You do not set any of these yourself.

At ingest, Rius reassembles the flattened message attributes into the GenAI message shape and stores them as gen_ai.input.messages and gen_ai.output.messages. The prompt and completion then render on the span’s Input / Output tab. A span that arrives without these attributes shows “Content not captured” there. Its timings, tokens and cost are unaffected.

The one setting this setup needs is the span attribute limit. OpenTelemetry keeps at most 128 attributes per span by default and drops the rest without an error. Because OpenInference writes one attribute per message field and per tool field, an agent loop with about ten tools passes that limit after six to nine turns. From then on the span silently loses attributes, such as the model, request parameters, tool definitions and system prompt in Python, or the usage, output and finish reason in Node. Raising the limit to 4096 keeps the span whole. The span attribute reference lists the full mapping.

Set it up

1. Wire the instrumentor to an OTLP exporter pointed at Rius. This example already sets the limit, through SpanLimits(max_span_attributes=4096), so you add nothing more:

from opentelemetry.sdk.trace import SpanLimits, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
 
# Raise the span attribute limit, so long agent loops keep every attribute
provider = TracerProvider(span_limits=SpanLimits(max_span_attributes=4096))
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="https://ingest.eu.console.rius-glassflow.com/v1/traces",
            headers={"Authorization": "Bearer <your API key>"},
        )
    )
)
OpenAIInstrumentor().instrument(tracer_provider=provider)

2. Give this exporter the full https://ingest.eu.console.rius-glassflow.com/v1/traces path. Use the host for your workspace’s region. In Node, use @opentelemetry/exporter-trace-otlp-proto.

3. If you did not use the code above, for example because your provider is created elsewhere or you work in another language, set the limit through the environment instead. Set OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT to 4096 in the process that runs the instrumentor. This works for any language.

4. Run one traced model call, open Traces in the console, open the new trace and select the model span. Its Input / Output tab should show the prompt and the reply. If the model name or messages are missing, check that the limit took effect, because a span over 128 attributes loses them.

The OpenInference page covers this sender in full, and Trace LLM calls with OpenTelemetry compares it with the other ways to send traces.

What you get

• The prompt and response of each model call on the span’s Input / Output tab.
• Generic input and output from input.value and output.value on any span the instrumentor records.
• Model, token usage and server-side cost on the same spans, because Rius normalizes the OpenInference model and token attributes onto their gen_ai.* equivalents. Rius computes cost from the model name and token counts, so you never send a cost.
• Searchable content under the canonical keys. Search and filter on gen_ai.usage.input_tokens, not on llm.token_count.prompt, because the original OpenInference keys are removed once they are mapped.
• No data loss on long agent loops, once the limit is at 4096.

Without the Rius SDK there are no Rius privacy controls in your process. Everything the instrumentor records, including the full request-parameter bag, is sent as recorded.

When something else fits

• If your app is Python or TypeScript and you want the same instrumentors without wiring a provider, use the Rius SDK. It ships them as integrations, sets the limit to 4096 and adds export-time privacy controls.
• If you want to keep prompts and replies out of Rius, use the SDK with RIUS_CAPTURE_CONTENT set to false, which strips content at export. Without the SDK, OpenInference’s own settings decide what is recorded: content is on by default, and OPENINFERENCE_HIDE_INPUTS or OPENINFERENCE_HIDE_OUTPUTS set to true hides it.
• If another sender already emits your traces, such as OpenLLMetry or a Collector, use that sender’s own setup. Rius reads its content attributes too.

Related