How do I set the model name attribute on an OpenTelemetry span in Python?

Rius reads the model name from the gen_ai.request.model attribute, which you set in Python with span.set_attribute on the span for the model call. Set it before you call the model, and add gen_ai.response.model afterwards if the provider reports a different one. Rius then shows the call in its per-model analytics and, with token counts, computes its cost.

How it works

The OpenTelemetry GenAI convention key for the model you ask for is gen_ai.request.model. It is a string, such as gpt-4o. Rius uses it in three places:

• Model analytics and cost. Cost is computed on the Rius side from gen_ai.request.model plus the gen_ai.usage.* token counts. A span needs the model and at least one token count. A span with no model, or a model that Rius’s price catalog does not know, gets an empty cost.
• Counting LLM calls. The console and alerts count a span as an LLM call when its kind is LLM, set directly or derived from a chat, text_completion or generate_content operation. A span with neither a kind nor an operation counts when it carries gen_ai.request.model. Agent, tool and retrieval spans are not LLM calls even when they carry a model.
• Search and filters. The value is stored under the key you sent, so filter on gen_ai.request.model in trace inspection.

gen_ai.response.model is a separate key for the model the provider reports back. Setting both lets you see when the provider served something other than the model you requested. Never send a cost attribute. Send the model name and the token counts.

Full key tables are in the span attribute reference.

Set it up

1. Register a tracer provider with the OTLP HTTP exporter once at startup. The exporter reads the standard OTEL_EXPORTER_OTLP_* variables. The vanilla OpenTelemetry SDKs page covers the endpoint, headers and install in full.

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
 
# With no arguments, the exporter reads the OTEL_EXPORTER_OTLP_* variables
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

2. Start a span for the model call and name it the way the GenAI conventions ask: the operation, then the model.

3. Set the model name with set_attribute, together with the operation and provider, before you call the model. Then add the token counts and the response model once the call returns:

from opentelemetry import trace
 
tracer = trace.get_tracer("my-agent")
 
with tracer.start_as_current_span("chat gpt-4o") as span:
    span.set_attribute("openinference.span.kind", "LLM")
    span.set_attribute("gen_ai.operation.name", "chat")
    span.set_attribute("gen_ai.request.model", "gpt-4o")
    span.set_attribute("gen_ai.provider.name", "openai")
 
    response = call_the_model()  # your own model call
 
    span.set_attribute("gen_ai.usage.input_tokens", response.usage.prompt_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", response.usage.completion_tokens)
    span.set_attribute("gen_ai.response.model", response.model)

4. Keep the model name an exact string, the same one you pass to the provider. Cost lookup depends on Rius recognizing it.

5. If you wrap each agent run in a root span, nest the model span inside it with the normal parent context, and the spans arrive as one trace.

What you get

• The span counts as an LLM call, and it appears in the per-model breakdown.
• With the token counts set, Rius computes the cost of the call from the model and tokens, as a list-price estimate in US dollars.
• You can search and filter traces on gen_ai.request.model, and compare it with gen_ai.response.model on the same span.
• Setting openinference.span.kind or gen_ai.operation.name makes Rius derive the other, so the span is classified in trace views.

For the full list of keys that drive analytics, see the span attribute reference. For the wider picture of sending LLM traces over OpenTelemetry, see Trace LLM calls with OpenTelemetry.

When something else fits

• If an OpenInference instrumentation already emits your spans, you do not need to set the key yourself. Rius fills gen_ai.request.model from llm.request.model_name, the model invocation parameter, or llm.model_name, in that order, so filter on gen_ai.request.model, not on llm.model_name.
• If you only need the model on a span that is not an inference call, such as a tool or retrieval step, the model is stored with the span but the span is not counted as an LLM call.

Related