How do I set the prompt cache token attribute on an OpenTelemetry span in JavaScript?

Rius reads prompt cache tokens from gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_write.input_tokens, which you set with span.setAttributes from @opentelemetry/api. Set them on the span for the model call, next to the model name and the input and output token counts. Rius prices each count at its own rate when it computes the span’s cost. You never send a cost value.

How it works

Rius has two keys for prompt cache tokens, both integers:

• gen_ai.usage.cache_read.input_tokens: prompt tokens served from the cache.
• gen_ai.usage.cache_write.input_tokens: prompt tokens written to the cache.

Both are subsets of gen_ai.usage.input_tokens, which counts prompt tokens including the cached ones. So send the full prompt count in gen_ai.usage.input_tokens and the cache counts beside it. Do not subtract the cached tokens from the input count.

Rius computes cost on its servers from gen_ai.request.model and the gen_ai.usage.* counts: input, output, cache read and cache write, each at its own rate. A span needs the request model and at least one count, and any count you leave out is read as 0. A span with no model, a model the price catalog does not know, or none of the counts gets no cost.

If an instrumentation already writes the older spelling gen_ai.usage.cache_creation.input_tokens for cache writes, Rius moves it to gen_ai.usage.cache_write.input_tokens at ingest. For new code, use cache_write.

Set it up

1. Register an OpenTelemetry tracer provider that exports to Rius over OTLP/HTTP. In Node.js, shut the provider down before the process exits, or queued spans are never sent. The vanilla OpenTelemetry SDKs page has the full Node.js setup, and the guide Trace LLM calls with OpenTelemetry covers the wider picture.

2. Start a span for the model call and set the attributes that identify it: the span kind, the operation, the request model and the provider.

3. After the call returns, set the token counts, including the two cache counts, and end the span:

import { trace } from '@opentelemetry/api'
 
const tracer = trace.getTracer('my-agent')
 
const span = tracer.startSpan('chat gpt-4o')
span.setAttributes({
  'openinference.span.kind': 'LLM',
  'gen_ai.operation.name': 'chat',
  'gen_ai.request.model': 'gpt-4o',
  'gen_ai.provider.name': 'openai',
})
const response = await callTheModel()
 
// cacheReadTokens and cacheWriteTokens are your own variables,
// read from the usage data your model provider returns
span.setAttributes({
  'gen_ai.usage.input_tokens': response.usage.prompt_tokens,
  'gen_ai.usage.output_tokens': response.usage.completion_tokens,
  'gen_ai.usage.cache_read.input_tokens': cacheReadTokens,
  'gen_ai.usage.cache_write.input_tokens': cacheWriteTokens,
  'gen_ai.response.model': response.model,
})
span.end()

4. Send integers. If your provider reports no cache writes, leave gen_ai.usage.cache_write.input_tokens out, and Rius reads it as 0.

5. If your spans carry many attributes, such as messages and tools spread over one attribute per field, set OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT=4096. OpenTelemetry SDKs keep 128 attributes per span by default and drop the rest without an error. The span attribute reference lists every key Rius reads.

What you get

• Cache-aware cost. Cache reads and cache writes are priced at their own rates, so the span’s cost reflects caching. Costs are list-price estimates in US dollars from LiteLLM’s public model-price catalog, not billed amounts.
• Searchable counts. The values are stored under the keys you sent. Search and filter on gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_write.input_tokens. A filter on an OpenInference key such as llm.token_count.prompt matches nothing, because Rius moves those values to the gen_ai.usage.* keys.
• Model analytics. A span with the model and token counts is a full generation. It appears in the per-model cost breakdown and in the cost totals.

When something else fits

If you use the Rius TypeScript SDK or an auto-instrumentation integration, the SDK fills in token counts for you, and you do not set these keys by hand. If an OpenInference or OpenLLMetry instrumentation already emits your usage, Rius normalizes its keys at ingest, so you only set the cache keys yourself on spans you create manually.

Related