How do I get LLM cost with the OpenTelemetry Python SDK?

Rius computes LLM cost from the model name and token counts on each span, so with the OpenTelemetry Python SDK you send those attributes and never a cost value. The SDK does not set them on its own. An instrumentor such as OpenInference sets them for you, and otherwise you add them to the span by hand. Rius prices each call and shows the cost on the span in the console.

How it works

The OpenTelemetry Python SDK creates and exports spans. It does not know about models or tokens, so a span you create yourself carries no cost inputs until you add them. Rius reads these attributes on each model call span:

• gen_ai.request.model: the requested model.
• gen_ai.usage.input_tokens: prompt tokens, including cached ones.
• gen_ai.usage.output_tokens: completion tokens, including reasoning ones.
• gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_write.input_tokens: optional, each priced at its own rate.

A span needs the request model and at least one token count. A count you leave out is read as 0. A span with no model, a model Rius’s price catalog does not know, or no counts gets an empty cost, not zero. Never send a cost attribute.

Whether your setup already sets these depends on what creates the span:

• A hand-made span: nothing is set until you add it.
• The OpenInference OpenAI instrumentor: it emits llm.* attributes, and Rius maps them onto the keys above at ingest. For example, llm.token_count.prompt and llm.token_count.completion fill the token counts. You add nothing.
• OpenLLMetry: Rius also maps its older usage spellings onto the current keys at ingest.

Costs are list-price estimates in US dollars, priced from LiteLLM’s public model-price catalog. They are not billed amounts.

Set it up

1. Register a tracer provider that exports to Rius. This is the shared setup, covered in full in Vanilla OpenTelemetry SDKs. Install opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http, then set OTEL_EXPORTER_OTLP_ENDPOINT to https://ingest.eu.console.rius-glassflow.com and OTEL_EXPORTER_OTLP_HEADERS to an authorization Bearer header with your API key. The protocol comes from the exporter you import from opentelemetry.exporter.otlp.proto.http, so OTEL_EXPORTER_OTLP_PROTOCOL is not needed here. These variables do nothing until this provider exists:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
 
# With no arguments, the exporter reads the OTEL_EXPORTER_OTLP_* variables
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

2. If an instrumentor creates your model call spans, such as OpenInference’s OpenAI instrumentor, make the model call and go to step 4. The instrumentor already emits the model and token counts. The OpenTelemetry tracing guide shows the OpenInference setup.

3. If you create the span yourself, set the model before the call and the token counts after it. Replace call_the_model and the response fields with your own client’s:

from opentelemetry import trace
 
tracer = trace.get_tracer("my-agent")
 
with tracer.start_as_current_span("chat gpt-4o") as span:
    span.set_attribute("openinference.span.kind", "LLM")
    span.set_attribute("gen_ai.operation.name", "chat")
    span.set_attribute("gen_ai.request.model", "gpt-4o")
    response = call_the_model()
    span.set_attribute("gen_ai.usage.input_tokens", response.usage.prompt_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", response.usage.completion_tokens)

4. If spans lose their model or tokens, raise OpenTelemetry’s default limit of 128 attributes per span by setting OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT to 4096. Instrumentors that flatten messages into one attribute per field pass 128 within a few agent turns.

5. Run one traced call, open Traces in the console, and open the new trace. The model call span shows the model name, token counts and cost. The span attribute reference lists every key Rius reads.

What you get

• A cost on each model call span, computed by Rius from the model and token counts, with cache reads and writes priced at their own rates.
• Cost totals and a per-model breakdown across your traces.
• Embedding calls count toward the per-model breakdown and cost totals, even though they are not counted as LLM calls.
• Estimates you can read as list prices in US dollars, never as exact billed amounts.

When something else fits

• If you would rather not set attributes yourself, the Rius SDK sets up the tracer provider and auto-instruments common LLM libraries, so model and tokens arrive without extra code.
• If you run a model that is not in the public price catalog, or you pay negotiated or discounted rates, Rius leaves the cost empty or shows the list price. It does not support custom prices.

Related