How do I set the token usage attribute on an OpenTelemetry span in Python?

Rius reads token usage from the gen_ai.usage.input_tokens and gen_ai.usage.output_tokens attributes, which you set as integers on the span with the OpenTelemetry Python API. Set them after the model call returns, together with gen_ai.request.model, and Rius shows the tokens on the span and computes the cost from them. You never send a cost value yourself.

How it works

Rius expects the OpenTelemetry GenAI convention keys for token counts. Each one is an integer on the span for the model call:

• gen_ai.usage.input_tokens: prompt tokens, including cached ones.
• gen_ai.usage.output_tokens: completion tokens, including reasoning ones.
• gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_write.input_tokens: prompt-cache reads and writes. Each is a subset of the input count.
• gen_ai.usage.reasoning.output_tokens: reasoning tokens, a subset of the output count.

Because the cache and reasoning counts are subsets, do not subtract them from the input and output counts before you send them.

Rius computes cost on its side from gen_ai.request.model plus these counts, with input, output, cache read and cache write each priced at their own rate. A span needs the request model and at least one count, and a count you leave out is read as 0. A span with no model, a model the price catalog does not know, or no counts gets no cost. Costs are list-price estimates in US dollars.

If another library already sets its own token keys, Rius maps them for you at ingest. OpenInference’s llm.token_count.prompt and llm.token_count.completion, and the older OpenLLMetry spelling gen_ai.usage.prompt_tokens, are moved onto the keys above. When you set the attributes by hand, use the gen_ai.usage.* keys directly. Searches and filters match gen_ai.usage.input_tokens, not the older spellings.

Set it up

1. Register a tracer provider with the OTLP HTTP exporter that points at your Rius ingest endpoint and API key. The vanilla OpenTelemetry SDKs page has the full setup, including the attribute limit you should raise for long LLM spans.
2. Open a span for the model call and set what you know before the call: the span kind, the operation, the request model and the provider.
3. Call the model, then set the token counts on the same span as plain Python integers. Here call_the_model() and response stand for your own code and its result:

from opentelemetry import trace
 
tracer = trace.get_tracer("my-agent")
 
with tracer.start_as_current_span("chat gpt-4o") as span:
    span.set_attributes({
        "openinference.span.kind": "LLM",
        "gen_ai.operation.name": "chat",
        "gen_ai.request.model": "gpt-4o",
        "gen_ai.provider.name": "openai",
    })
    response = call_the_model()
    span.set_attributes({
        "gen_ai.usage.input_tokens": response.usage.prompt_tokens,
        "gen_ai.usage.output_tokens": response.usage.completion_tokens,
        "gen_ai.response.model": response.model,
    })

4. If your provider reports cache or reasoning tokens, add gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_write.input_tokens or gen_ai.usage.reasoning.output_tokens in the same call.
5. Nest the span under a single root span for the agent run, so the calls arrive as one trace. The span attribute reference lists every key Rius reads, and the guide Trace LLM calls with OpenTelemetry covers the wider path.

What you get

• The token counts on the span for each model call, stored under the keys you set.
• A cost for that call, computed by Rius from the model and the counts, with cache reads and writes priced separately.
• The span in model analytics and in the per-model cost breakdown, because it carries a model and counts.
• A span that counts as an LLM call, because its kind is LLM and it carries the request model.
• Spans from other OpenTelemetry libraries in the same trace get their token keys normalized onto the same canonical names, so your counts and theirs read alike.

When something else fits

• If you use the Rius Python SDK or one of its integrations, you do not set these attributes yourself, because the instrumentation sends token counts for you.
• If a library already emits OpenInference or OpenLLMetry spans, leave its keys as they are and let Rius map them at ingest.
• If your span has no model name, set gen_ai.request.model first, because without it Rius cannot compute a cost.

Related