How do I alert on input tokens for a single model?

Rius lets you alert on input tokens for one model with a custom alert that has a Model filter and an Input tokens rule. The Model filter matches gen_ai.request.model and applies before the rule is calculated, so only spans for that model feed it. Input tokens is measured per trace, so a rule such as p95(Input tokens) > 20000 compares against the per-trace figure, not the workspace total.

How it works

A custom alert on the custom alerts page is made of optional filters, one or more rules and one shared rolling window. Filters run first, then the rules aggregate what is left.

• The filter this needs. Use the Model filter. It matches gen_ai.request.model, the model name your instrumentation records on each span. Text filters offer =, ≠, in and not_in, and the builder suggests values from your workspace’s live data. Use in to cover several models in one alert, or not_in to alert on every model except some.
• Filters match each span’s own value. A span is kept if it carries the model you picked, and only kept spans feed your rules.
• The metric. Input tokens is a numeric metric in tokens. It offers avg, min, max, p50, p95 and p99.
• Per-trace aggregation. Numeric metrics use per-trace semantics. avg(Input tokens) is the average input tokens per trace in the window, not the total across the workspace. Pick max or p95 to catch a few oversized traces that an average hides.
• Per-call metrics are different. Time to first token, Context usage and LLM calls are measured per LLM call, so their aggregations run over calls, not traces. Input tokens is not one of them.
• Comparison and window. Use <, <=, >, >= or == against a token count. The window runs from 5 minutes to 24 hours.

If you care about how full the model’s context window is rather than a raw token count, Context usage is the per-call metric for that. It divides one call’s input tokens, prompt cache included, by the model’s window, and you compare it against a value between 0 and 1.

Set it up

1. Open the custom alert builder from Alerts and start a new custom alert.
2. Add a filter: Model, operator =, and pick your model from the suggested values. Use in if the alert should cover more than one model.
3. Add a rule: metric Input tokens, an aggregation (for example p95), an operator and your threshold in tokens.
4. Set the window, for example 15 minutes. All rules in the alert share it.
5. Check the plain-language preview. It states exactly when the alert fires and becomes the alert’s description.
6. Attach a channel (webhook, email or Slack) and choose a Renotify policy, off or every N minutes.

The alert from steps 2 to 4 reads like this in the rule grammar. The model name and the threshold are examples, so replace them with your own:

FILTERS   Model = claude-haiku-4-5-20251001
RULES     p95(Input tokens) > 20000
WINDOW    15 minutes

To require a second condition, add another rule. Rules combine with AND only, and all of them must hold in the window for the alert to fire. For either-or behavior, such as two different thresholds, create a second alert. An alert takes up to 10 filters with up to 50 values each, and up to 10 rules.

What you get

• An alert scoped to one model, so a heavy model does not mask a quiet one.
• A firing record that shows the observed value of every rule against its threshold, so you can see which condition tripped.
• A details page with the generated description, the editable rule, the attached channels and the firing history.
• Notifications through webhook, email or Slack, with optional renotify every N minutes while the alert keeps firing.
• An alert you can edit or delete later. Deleting it removes its firing history after you confirm.

When something else fits

• If you want to watch cost or context fill rather than input tokens, build the same alert with the Cost or Context usage metric. Cost only counts spans whose model has a known price.
• If you want one alert across all models, leave the Model filter out and filter by Service or Agent instead.

Related