How do I alert on time to first token for one service?

Rius lets you alert on time to first token for one service with a custom alert that has a Service filter and a Time to first token rule, such as p95 over 800 ms. The filter limits the rule to spans from that service, and the rule is measured per LLM call, so the p95 is taken over calls, not traces. The alert fires when the calls in the rolling window cross your threshold.

How it works

A custom alert is one or more rules combined with AND, all evaluated over one shared rolling window. Optional filters select which spans feed the rules before any aggregation. For this question you need one filter and one rule:

• Filter: Service. It matches service.name from your instrumentation. Text fields offer =, ≠, in and not_in, and the values are suggested from your workspace’s live data. Each filter matches a span’s own value, so every span from that service feeds the rule.
• Rule: Time to first token. It is a numeric metric in milliseconds with the aggregations avg, min, max, p50, p95 and p99.

Most numeric metrics aggregate per trace. avg(Cost) is the average cost per trace in the window, not the workspace total. Time to first token is an exception, along with Context usage and LLM calls: it is measured per LLM call. An LLM call is an inference call, a span the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count, even when they carry a model. So p95(Time to first token) means the 95th percentile across the LLM calls of that service in the window.

Service is the right filter here because it measures whole runs. Step, Model, Span kind and Operation filters change what a metric covers, so use them only when you want to narrow the rule to one step or model.

Time to first token exists only on streamed LLM calls whose instrumentation records it. Calls without a value are left out of the aggregation.

Set it up

1. Open the custom alert builder from Alerts and start a new custom alert.
2. Add a filter: field Service, operator =, and pick your service name from the suggested values. To cover several services in one alert, use in and list them. They are then measured together, not separately.
3. Add a rule: metric Time to first token, an aggregation, a comparison operator and a value in milliseconds. Use p95 or p99 to catch slow tails, and avg to watch the typical call.
4. Set the window. It is shared by all rules and runs from 5 minutes to 24 hours.
5. Check the plain-language preview. It states exactly when the alert will fire, and it becomes the alert’s description on its details page.
6. Attach a channel (webhook, email or Slack) and save.

The alert you build looks like this:

FILTERS   Service = checkout-agent
RULES     p95(Time to first token) > 800
WINDOW    15 minutes

The service name and the 800 ms threshold are example values. Replace them with your own. The comparison operators are less than, less than or equal, greater than, greater than or equal, and ==.

To add a second condition, such as an error rate, add another rule. Rules combine with AND only, so the alert fires only when every rule is true. If you want either-or behavior, create a second alert. An alert takes up to 10 filters with up to 50 values each, and up to 10 rules. The full reference is on Custom alerts.

What you get

• An alert scoped to one service, so slow first tokens in one agent do not mix with the rest of your workspace.
• A threshold on the calls themselves. A p95 rule fires when the slowest 5% of that service’s LLM calls in the window are over your limit, even if the average looks fine.
• A details page with the generated description, the editable rule, attached channels and firings. Each firing records the observed value of every rule against its threshold, so you can see which condition tripped.
• Coverage on firings. Firings on Time to first token show the coverage ratio, so you can tell a real reading from a partly covered one. A rule held under its threshold by low coverage stays quiet, so confirm your calls are streamed and report a first token before you rely on the alert.
• The same metric as a summary elsewhere. Time to first token (p50 and p95) also appears in Usage and cost, so you can pick a threshold from what you already see.

When something else fits

• If you want a time to first token threshold on one step or one model inside a service, add a Step or Model filter next to the Service filter. Then the rule measures only the matched spans.
• If your calls are not streamed, or their instrumentation records no first token, this metric has nothing to measure. Alert on Duration instead, which measures whole runs when you filter by Service or Agent.
• If you want one alert per service with different thresholds, create a separate custom alert for each service.

Related