Skip to Content

What does the gen_ai.response.time_to_first_chunk attribute mean in Rius?

In Rius, gen_ai.response.time_to_first_chunk holds how many seconds passed between the start of a streaming model call and the arrival of its first content chunk. Rius derives time to first token from it, so you see how fast streamed responses begin. The key is a double, and you set it once per streaming call, together with gen_ai.request.stream set to true.

How it works

  • What the key holds. A double, in seconds, measured from the span start to the first streamed chunk. It is not a timestamp and it is not the total duration of the call. The OpenTelemetry GenAI conventions define it, at Development stability, as the time from the start of the inference operation to the first chunk of its streaming response, where a chunk is a provider stream event with new, non-empty output. Its registry entry links to the upstream definition.
  • Which spans carry it. The span for a streaming model call. A span that does not stream has no first chunk, so it has no value to send.
  • What Rius computes. Rius places the first chunk at the span start plus this duration, never later than the span’s end. Time to first token comes from that point.
  • When a producer sends its own event. If the span carries OpenInference’s First Token Stream Event or the Vercel AI SDK’s ai.stream.firstChunk, the time of that event is the first chunk. In that case Rius does not use gen_ai.response.time_to_first_chunk for the span. When you export through the Rius SDKs, they also derive the duration from such an event for you.
  • The older signal. Earlier Rius SDK releases also wrote a span event named gen_ai.first_token beside the attribute. It is deprecated. Rius still reads it and still uses that name for the first-chunk event it stores, but new instrumentation should send only the attribute. For a span with no first-chunk event, Rius creates the stored event itself from the attribute. The deprecated key is listed in the gen_ai registry page, and the full attribute contract is in the span attribute reference.

Set it up

  1. Make sure your tracing already reaches Rius. If you send spans without the Rius SDK, the Trace LLM calls with OpenTelemetry guide covers the exporter setup.
  2. If you use the Rius SDK, call record_first_token() when the first content chunk arrives. It sets gen_ai.response.time_to_first_chunk once, in seconds from the span start, together with gen_ai.request.stream set to true. In TypeScript the call is recordFirstToken(). No event is needed.
  3. If you instrument the call yourself with OpenTelemetry, set the same two attributes on the span for the streaming call when the first content chunk arrives. The value is your own measured delay in seconds:
gen_ai.request.stream: true
gen_ai.response.time_to_first_chunk: 0.42
  1. Set the attribute once per call. Do not also send the gen_ai.first_token event.
  2. Open the trace in Rius and check the time to first token on the span of the streamed call.

What you get

  • Time to first token for each streaming model call, derived from one number on the span.
  • Time to first token in your usage and cost view as p50 and p95, so you can watch how quickly streamed answers start over a window from 15 minutes to 30 days.
  • Time to first token as a measure for custom alert rules, so you can be told when streamed answers start slowly.
  • Time to first token (p50) in the comparison of two agent versions, next to failed rate, cost per trace and latency.
  • The same result whether the span comes from the Rius SDKs or from a plain OpenTelemetry setup, because the attribute is the signal Rius reads when no producer event is present.

When something else fits

  • If your instrumentation already records a first-chunk event, such as OpenInference’s or the Vercel AI SDK’s, you do not need to set this attribute. Rius uses the event’s own time.
  • If a call does not stream, leave the attribute out. Use the span’s duration for its total latency.

Still stuck? Get help

Last updated on
Quickstart