How do I trace LLM calls with OpenTelemetry in Python?

Rius accepts traces from the standard OpenTelemetry Python SDK, so you point its exporter at the Rius OTLP endpoint with environment variables and your LLM calls arrive as generation spans. Each span carries the model, token usage and messages, and Rius computes the cost on its side. You give up the conveniences the native Rius SDK adds on top, such as message normalization, a raised attribute limit and auto-instrumentation. The wider options are covered in Trace LLM calls with OpenTelemetry.

How it works

The Rius ingest endpoint is plain OTLP over HTTP, in protobuf or JSON. There is no gRPC endpoint, and Rius ingests traces only, not OTLP metrics or logs. Your API key goes in an Authorization: Bearer header, and the key decides which workspace the traces land in.

Rius reads the OpenTelemetry GenAI attributes (gen_ai.*). It also normalizes the OpenLLMetry and OpenInference conventions onto them at ingest. A call traced by any of those instrumentors therefore gets model, token and cost analytics. Cost is a list-price estimate in US dollars, and you never send a cost or latency value yourself.

The Python OpenTelemetry SDK reads the standard exporter variables once you register a tracer provider with an OTLP exporter, so configuring it needs no Rius-specific code. The ingest host carries your workspace’s region. The examples here use the eu host, https://ingest.eu.console.rius-glassflow.com.

Set it up

1. Create an API key in the Rius console. It starts with ri_.
2. Set the standard exporter variables in the environment of your Python process. Set the protocol explicitly, and raise the attribute limit so long LLM spans stay whole, because the default of 128 attributes is too low:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.eu.console.rius-glassflow.com"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer <your API key>"
export OTEL_SERVICE_NAME="my-agent"
# Keep long LLM spans whole; the default of 128 attributes is too low
export OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT=4096

Then register a tracer provider with an OTLP exporter, which reads them:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
 
# With no arguments, the exporter reads the OTEL_EXPORTER_OTLP_* variables
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

3. Emit a well-formed LLM span. Start the span with these attributes:

   • openinference.span.kind set to LLM
   • gen_ai.operation.name set to chat
   • gen_ai.request.model, for example gpt-4o
   • gen_ai.provider.name, for example openai

   After the model call returns, add gen_ai.usage.input_tokens, gen_ai.usage.output_tokens and gen_ai.response.model. Also add gen_ai.output.messages as a JSON string of messages shaped like role plus parts, where each part has a type and content. Model, tokens and messages make the span a full generation, with cost computed by Rius and a place in model analytics. Do not send a cost attribute.

4. Nest spans the normal OpenTelemetry way, with parent context, and they arrive as one trace. Wrap each agent run in one root span, because its name, duration and status are what run-level views show.

5. If you would rather not write spans by hand, use an instrumentor. This OpenInference setup for OpenAI sends to the full /v1/traces path, because this exporter does not append it for you:

from opentelemetry.sdk.trace import SpanLimits, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
 
provider = TracerProvider(span_limits=SpanLimits(max_span_attributes=4096))
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="https://ingest.eu.console.rius-glassflow.com/v1/traces",
            headers={"Authorization": "Bearer <your API key>"},
        )
    )
)
OpenAIInstrumentor().instrument(tracer_provider=provider)

6. Run one traced call, open Traces in the console at https://console.rius-glassflow.com, and open the new trace. The LLM span shows a model name, token counts and cost. A 401 means the API key is wrong or revoked. A span that arrives without a model or tokens usually lost attributes on the way, for example to a Collector processor that strips gen_ai.* keys or to the 128 attribute limit.

The full set of variables and the span walkthrough are on Vanilla OpenTelemetry SDKs.

What you get

• Each LLM call as a generation span in a trace, with model, token usage and messages.
• Cost per call computed by Rius from the model and token counts.
• Model analytics for every span that carries a model, tokens and messages.
• A configuration that stays the same if you later adopt the Rius SDK, because raw OpenTelemetry emission is fully supported permanently and code written against it keeps working unchanged.

What you give up against the native SDK is convenience, not a different protocol. The native SDK adds:

• message normalization
• mapping of third-party attribute dialects onto the canonical keys
• a raised attribute limit
• idempotent first-token capture
• export-time privacy controls
• the auto-instrumentation integrations

With raw OpenTelemetry you set the attribute limit yourself, as shown above, and you write or choose the instrumentation.

When something else fits

• If you have no tracing yet and want the least setup, the Rius SDK with a library extra such as pip install "glassflow-rius[openai]" openai and one init() call auto-instruments that library.
• If you already run an OpenTelemetry Collector, add the otlphttp exporter pointing at Rius instead of changing application code.

Related