How do I alert on reasoning tokens for one service?

Rius lets you alert on reasoning tokens for one service with a custom alert that has a Service filter and a rule on the Reasoning tokens metric. The filter keeps only that service’s spans, and the rule compares an aggregation such as avg or p95 against your threshold over a rolling window. Reasoning tokens are counted per trace, so avg(Reasoning tokens) is the average per trace in the window, not a total.

How it works

A custom alert is one or more rules, combined with AND, evaluated over one shared rolling window. Filters apply first and select which spans feed the rules. Aggregation comes after.

For this question, two parts matter:

• The filter. Service matches service.name from your instrumentation. It matches each span’s own value, so every span from that service feeds the rule, not only root spans. That makes it the right scope for whole runs.
• The metric. Reasoning tokens is a numeric metric. You can aggregate it with avg, min, max, p50, p95 or p99, and the unit is tokens.

Numeric metrics use per-trace aggregation. avg(Reasoning tokens) is the average reasoning tokens per trace in the window, and p95(Reasoning tokens) is the 95th percentile across traces. Time to first token, Context usage and LLM calls are the exceptions, because they are measured per LLM call. Reasoning tokens is not one of them.

Reasoning tokens live on model-call spans (LLM, embedding and reranker). A Service filter includes those spans along with the rest of the service’s spans, so the metric reads correctly. Be careful with Step, Span kind or Operation filters. If they select only tool, chain or agent spans, the rule measures no tokens. Use Service or Agent to measure whole runs.

Set it up

1. Create a custom alert in Alerts.
2. Add a filter on the Service field with = and your service name. Rius suggests values from your workspace’s live data. To cover several services in one alert, use in with a list of names.
3. Add a rule on Reasoning tokens. Pick an aggregation, a comparison operator and a threshold. Use avg to watch the typical trace, or p95 to catch the heaviest 5% of traces.
4. Set the window. It is shared by all rules and can be between 5 minutes and 24 hours.
5. Check the plain-language preview. It states exactly when the alert fires, and it becomes the alert’s description.
6. Attach a channel (webhook, email or Slack) so firings reach you.

The finished alert reads like this. The service name and the threshold are examples, so use your own:

FILTERS   Service = checkout-agent
RULES     avg(Reasoning tokens) > 20000
WINDOW    15 minutes

This fires when the average trace from checkout-agent in the last 15 minutes uses more than 20,000 reasoning tokens.

To add a second condition, add another rule. Rules combine with AND only, so the alert fires only when every rule is true. If you want either-or behavior, create a second alert. One alert takes up to 10 filters with up to 50 values each, and up to 10 rules.

The full builder reference is on Custom alerts.

What you get

• An alert scoped to one service, so other services’ reasoning usage never trips it.
• A threshold on a value you choose: the average, a percentile or the maximum across traces.
• A details page with the generated description, the editable rule, the attached channels and the firings.
• Firings that record the observed value of every rule against its threshold, so you can see which condition tripped.
• A rule you can edit later, or delete. Deleting a custom alert removes its firing history after you confirm.

When something else fits

• If you need either-or conditions, create separate alerts, because rules combine with AND only.
• If you only want to see token trends rather than be paged, Usage and cost on Home summarizes tokens over time. Reasoning tokens for one call show in that model call’s span details in the trace view.

Related