How do I alert on input tokens for one span kind?

Rius lets you alert on input tokens for one span kind by adding a Span kind filter to a custom alert with a rule on the Input tokens metric. The filter selects which spans feed the rule before anything is aggregated. Input tokens is a per-trace metric, so avg and p95 are computed per trace in the window, not per call and not as a workspace total.

How it works

A custom alert in Rius is one or more rules, combined with AND, evaluated over one shared rolling window. You can scope it with filters. Filters select which spans feed the rules, before any aggregation. Each filter matches a span’s own value.

For this question you need two things:

• The Span kind filter. Its values are LLM, tool, chain, retriever, and so on. Text filters offer =, ≠, in and not_in, and values are suggested from your workspace’s live data. Use = for one kind, or in for a short list.
• A rule on Input tokens. It is a numeric metric measured in tokens. The aggregations are avg, min, max, p50, p95 and p99, and the comparison operators are <, <=, >, >= and ==.

Rules measure only the matched spans. Input tokens and cost are recorded on LLM spans, so a Span kind filter of tool, chain or agent measures no tokens and no cost. Use LLM, or leave the filter off.

How the metric aggregates matters for the threshold you pick. Numeric metrics use per-trace semantics, so avg(Input tokens) is the average input tokens per trace in the window. Time to first token, Context usage and LLM calls are the exception, because they are measured per LLM call. Input tokens is not one of them, so a threshold on it describes traces, not individual calls.

An alert takes up to 10 filters with up to 50 values each, up to 10 rules, and a window between 5 minutes and 24 hours.

Set it up

1. Open the custom alert builder, described on the custom alerts page.
2. Add a filter on Span kind, choose =, and pick the kind, for example LLM. If you only want spans of one operation, the Operation filter on gen_ai.operation.name (such as chat, execute_tool or retrieval) is the other way to narrow spans.
3. Add a rule on Input tokens. Pick an aggregation, an operator and a threshold in tokens. A high percentile such as p95 catches the heaviest traces without being moved by a few small ones.
4. Set the window. It is shared by all rules and runs from 5 minutes to 24 hours.
5. Check the plain-language preview. It states exactly when the alert fires, and it becomes the alert’s description on its details page.
6. Attach a channel (webhook, email or Slack) and choose the Renotify policy: off, or every N minutes.

The alert as the builder expresses it, with example values you replace with your own:

FILTERS   Span kind = LLM
RULES     p95(Input tokens) > 20000
WINDOW    15 minutes

To add a second condition, add another rule. Rules combine with AND only, with no OR and no nesting, so for either-or behavior create a second alert. For example, add avg(Cost) > $0.50 to the same alert to fire only when heavy input and high cost occur together.

When you edit the alert later, the details page shows the generated description, the editable rule, the attached channels and the firings.

What you get

• An alert scoped to the one span kind you chose, so spans of other kinds do not feed the rule.
• A threshold in tokens with a choice of avg, min, max, p50, p95 or p99, read per trace over your window.
• A firing record that stores the observed value of every rule against its threshold, so you see which condition tripped.
• Delivery to webhook, email or Slack, with optional renotification.
• A custom alert you can edit or delete. Deleting removes its firing history after an explicit confirmation.

The Alerts section covers the firing lifecycle and the pre-defined alerts that sit alongside your custom ones.

When something else fits

• If you care how full the model’s context is for each call, use Context usage instead. It is measured per LLM call, for example p95(Context usage) > 0.8, and it leaves out calls whose model has no known context window.
• If you want to watch spend rather than size, use a Cost rule. It is only as complete as your pricing coverage, because spans whose model has no known price carry no cost.
• If you only need to know that input tokens are trending up, the same metrics appear as a summary on the usage and cost view rather than as a trigger.

Related