How do I alert on input tokens for a single agent?

Rius lets you alert on input tokens for one agent with a custom alert that has an Agent filter and a rule on the Input tokens metric. The filter narrows the spans before any aggregation, so the rule only sees that agent. You choose the aggregation (average, p95 and so on), the threshold and a rolling window, and the builder previews in plain language when the alert fires.

How it works

A custom alert on Custom alerts has three parts: optional filters, one or more rules and one shared window. Filters apply first, then the rules aggregate what is left.

• The filter this needs: Agent. It matches the agent identity Rius derives, as each span carries it. Pick the agent from the suggested values, which come from your workspace’s live data. Use = for one agent, or in to cover several. ≠ and not_in exclude agents instead.
• What the filter matches: each span’s own value. An Agent filter selects every span that carries the agent, not only the root span, so tool and model spans of that agent feed the rule too.
• A broader scope: Service matches service.name from your instrumentation. Use it when you want everything one service sends rather than one agent.
• The metric: Input tokens is a numeric metric. It offers avg, min, max, p50, p95 and p99, measured in tokens.
• How it aggregates: numeric metrics use per-trace semantics. avg(Input tokens) is the average per trace in the window, not the total for the agent. A rule of avg(Input tokens) above 8,000 fires when the typical trace is large, not when the agent’s overall token volume is high.
• Per-call metrics are different: Time to first token, Context usage and LLM calls are measured per model call, so their aggregations run over calls. Input tokens is not one of them.

The window is shared by all rules and runs from 5 minutes to 24 hours. An alert takes up to 10 filters with up to 50 values each, and up to 10 rules.

Set it up

1. Open the custom alert builder from Alerts and start a new custom alert.
2. Add a filter: field Agent, operator =, and the agent’s name chosen from the suggestions. If you don’t see the name, the agent hasn’t sent spans that carry it yet.
3. Add a rule: metric Input tokens, an aggregation, a comparison and a threshold. Use avg to catch a general rise per trace. Use p95 or max to catch a few oversized traces.
4. Set the window. A short window reacts quickly. A longer one smooths out single large traces.
5. Read the plain-language preview. It becomes the alert’s description on its details page, so check that it names the agent and the aggregation you intend.
6. Attach a channel (webhook, email or Slack) and choose a Renotify policy, off or every N minutes.

The alert you build reads like this. The agent name and numbers are examples, so use your own:

FILTERS   Agent = support-agent
RULES     p95(Input tokens) > 20000
WINDOW    1 hour

To require two conditions at once, add a second rule. Rules combine with AND only, and there is no OR. If you want either-or, create a second alert. For example, add count(Errors) > 10 to the same alert to fire only when large inputs and errors occur together.

What you get

• An alert that watches only the agent you named, so other agents in the workspace cannot trip it.
• A threshold in tokens, with the aggregation that fits the question: typical size with avg, the heavy tail with p95 or p99, the single worst trace with max.
• Firings that record the observed value of every rule against its threshold, so you can see which condition tripped.
• A details page with the generated description, the editable rule, attached channels and the firing history.
• The freedom to edit or delete the alert later. Deleting a custom alert removes its firing history after you confirm.

If you also care about how full the model’s context is, add a rule on Context usage. It is measured per call and compares against a value between 0 and 1.

When something else fits

• If you want one alert for several agents with different thresholds, create one alert per agent, because filters apply to every rule in an alert.
• If you only want to see token totals, not be paged, the usage view on Monitoring usage and cost shows the same metrics as a summary instead of a trigger.
• If you want a built-in pattern such as repeated errors, the pre-defined alerts already cover it and need no rules.

Related