How do I record token counts with OpenInference instrumentors?

Rius records token counts from OpenInference instrumentors with no extra attribute, because it maps their llm.token_count values onto gen_ai.usage values at ingest. The instrumentor already writes the counts. The one setting you add is a higher span attribute limit, so the counts are not dropped. Rius then computes cost from the model and those counts.

How it works

OpenInference instrumentors write token counts as llm.token_count.prompt and llm.token_count.completion, plus cache and reasoning details. Rius normalizes these at ingest onto the keys its analytics read:

• llm.token_count.prompt becomes gen_ai.usage.input_tokens.
• llm.token_count.completion becomes gen_ai.usage.output_tokens.
• Cache and reasoning details fill gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_write.input_tokens and gen_ai.usage.reasoning.output_tokens.

The model comes from llm.request.model_name, else the model member of llm.invocation_parameters, else llm.model_name. Cost needs the request model and at least one count. A count a span leaves out is read as 0, and a model Rius does not price gets no cost.

The setting that matters is the span attribute limit. OpenTelemetry keeps 128 attributes per span by default and drops the rest without an error. OpenInference writes one attribute per message field and per tool field, so an agent loop passes 128 within a few turns. In Node the usage is among the attributes lost, and in Python the model, request parameters and tool definitions go first. A span that loses its usage shows no token counts.

If an attribute is native gen_ai.*, it wins over a normalized value. Normalization only fills gaps.

Set it up

1. Check whether your instrumentor already sends counts. For the OpenAI instrumentor it does. On streamed Chat Completions calls, pass stream_options={"include_usage": True}, or the stream carries no usage to record. You only need an exporter with a raised attribute limit. This example sets the limit to 4096 on the tracer provider and sends spans to the Rius ingest endpoint:

from opentelemetry.sdk.trace import SpanLimits, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
 
provider = TracerProvider(span_limits=SpanLimits(max_span_attributes=4096))
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="https://ingest.eu.console.rius-glassflow.com/v1/traces",
            headers={"Authorization": "Bearer <your API key>"},
        )
    )
)
OpenAIInstrumentor().instrument(tracer_provider=provider)

2. Keep the full https://ingest.eu.console.rius-glassflow.com/v1/traces path on this exporter. It does not append the path itself, unlike the environment variables and the Collector. Use your workspace’s region in the host.
3. In another language, or when you cannot edit the provider, set the environment variable OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT to 4096 instead. It works for any OpenTelemetry SDK.
4. If your instrumentor sends no counts, set gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on the span yourself, along with gen_ai.request.model. Add the cache and reasoning keys if you have them. Send tokens and the model name, never a cost attribute, because Rius computes cost.
5. Run one call and open the trace. The model call’s span should show the model, input and output tokens and cost. For the full exporter options, see OpenInference and Trace LLM calls with OpenTelemetry.

If you use the Rius SDK instead, it ships these same instrumentors, sets the 4096 limit for you and applies the same mapping before export. In TypeScript, install the instrumentor package for your library, such as @arizeai/openinference-instrumentation-openai.

What you get

• Token counts on each model call’s span, under gen_ai.usage.input_tokens and gen_ai.usage.output_tokens.
• Cost per call, a list-price estimate in US dollars, computed from the model and counts, with cache reads and writes priced at their own rates.
• Search and filters that use the normalized key. A filter on gen_ai.usage.input_tokens matches, while one on llm.token_count.prompt matches nothing, because Rius removes the source key after mapping it.
• Model analytics on the same spans, and time to first token on streamed calls from the Python OpenAI instrumentor.

The full list of keys and the mapping rules is in the span attribute reference.

When something else fits

• If you want one call to set everything up, including the attribute limit, the Rius SDK wires the same instrumentors for you.
• If you emit spans by hand or from another convention, set the gen_ai.usage.* keys directly, since native keys always win.

Related