How do I alert on time to first token for one step?

In Rius, you create a custom alert with a Step filter set to the exact span name of the model call, then add a rule on the Time to first token metric. Rius measures time to first token per model call, so p95(Time to first token) > 2000 fires when the slowest 5% of that step’s calls in the window take longer than 2000 ms to start streaming. The alert evaluates only the spans the filter matches.

How it works

A custom alert is one or more rules combined with AND, evaluated over one shared rolling window. Optional filters pick which spans feed the rules, and they apply before any aggregation.

Two things matter for this question:

• The Step filter matches the span name exactly. Step is the span’s own name, so the value must be the name of the model-call span you care about. Values are suggested from your workspace’s live data. Unnamed model, tool, agent and retrieval spans from the 1.0.0 SDKs are named after their operation and target, such as execute_tool search_orders. If your agents upgraded from an earlier SDK, use the new name.
• Time to first token is measured per call, not per trace. Most numeric metrics, such as avg(Cost), average over traces. Time to first token, Context usage and LLM calls are the exception: their aggregations run over individual LLM calls. An LLM call is an inference call, a span the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count, even when they carry a model. So the Step you filter on should be a model-call span.

Time to first token offers avg, min, max, p50, p95 and p99, in milliseconds. Comparison operators include >, >= and ==. A percentile such as p95 is usually a better trigger than an average for a latency that spikes now and then.

Set it up

1. Open the custom alert builder and add a filter: field Step, operator =, and the span name of the model call. Use in if the step has more than one name, for example before and after an SDK upgrade.
2. Add a rule on Time to first token. Pick an aggregation (p95 is a good start), a comparison operator and a threshold in milliseconds.
3. Set the window. It can be between 5 minutes and 24 hours, and every rule shares it. A short window reacts fast but reads fewer calls, so pick one long enough to hold a meaningful number of calls for that step.
4. Check the plain-language preview. It states exactly when the alert fires and becomes the alert’s description on its details page.
5. Save, then attach channels (webhook, email or Slack) so firings reach someone.

The finished alert reads like this:

FILTERS   Step = answer-question
RULES     p95(Time to first token) > 2000
WINDOW    15 minutes

Here answer-question stands for your own span name. To alert on slow first tokens only when the step is also failing, add a second rule such as a count of Errors. Rules combine with AND only. For either-or behaviour, create a second alert.

What you get

• An alert scoped to one step, so a slow first token in a different step does not trip it.
• A threshold that reads over calls, so the percentile describes the model calls in the window, not whole traces.
• Firings that record the observed value of every rule against its threshold, so you can see which condition tripped.
• A coverage ratio on each firing. Time to first token exists only on streamed LLM calls whose instrumentation records it, so the ratio shows whether a reading comes from most calls or only a few. A rule held under its threshold by low coverage stays quiet, so confirm your step’s calls are streamed and recorded before you rely on the alert.
• Delivery through your attached channels, with an optional Renotify policy of off or every N minutes.

When something else fits

• To measure time to first token across every model call of a service or agent, filter by Service or Agent instead of Step. Those filters match every span that carries the value.
• If the step is not streamed, it records no time to first token. Use Duration for it instead. With a Step filter, Duration runs per trace from the earliest matched span’s start to the latest matched span’s end, not across the whole run.
• For the built-in alerts that need no setup, see Alerts.

Related