How do I alert on context window usage for a single model?

Rius lets you alert on context window usage for one model with a custom alert that filters Model to that model and puts a threshold on the Context usage metric. For example, p95(Context usage) > 0.8 fires when the fullest 5% of that model’s calls in the window fill more than 80% of its context window. Context usage is measured per LLM call, so the aggregation runs over calls, not traces.

How it works

A custom alert is one or more rules, combined with AND, evaluated over one shared rolling window. Filters apply first and select which spans feed the rules. Aggregation comes after.

The filter this needs. Use the Model filter. It matches gen_ai.request.model, so only spans for that model reach the Context usage rule. Text filters offer =, ≠, in and not_in, and values are suggested from your workspace’s live data. Use = for one model. Use in if you want one alert to cover a few models together.

What Context usage is. It is the share of the model’s context window that one LLM call filled: its input tokens, prompt cache included, divided by the model’s window. You compare it against a value between 0 and 1, and the builder shows it as a percentage. The aggregations are avg, min, max, p50, p95 and p99.

Per call or per trace. Most numeric metrics, such as Cost, the token counts and Duration, aggregate per trace. avg(Cost) is the average cost per trace in the window, not a workspace total. Context usage is an exception, along with Time to first token and LLM calls. These are measured per LLM call, so p95(Context usage) is a percentile over calls. An LLM call is an inference call, a span the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count, even when they carry a model.

Calls without a known window. A call whose model has no known context window has no context usage. It is left out of the aggregation and never counted as 0. Firings on Context usage show the coverage ratio, so you can tell a full reading from a partly covered one. A rule held under its threshold by low coverage stays quiet.

Which aggregation to pick. Use a high percentile such as p95 or p99 to catch the calls closest to the limit, since an average hides them. Use max if one overfull call matters. Use avg if you care about a general drift upward.

Set it up

1. Open the custom alert builder and add a filter: Model = the model you want to watch. The value is the model name your instrumentation reports.
2. Add a rule on Context usage. Choose an aggregation and a threshold between 0 and 1. For 80%, enter 0.8.
3. Set the window. It can be anywhere from 5 minutes to 24 hours. A short window reacts fast, and a longer one smooths out single large prompts.
4. Read the plain-language preview. It states exactly when the alert fires, and that sentence becomes the alert’s description.
5. Attach a channel (webhook, email or Slack) and choose a Renotify policy, off or every N minutes.

The finished alert reads like this:

FILTERS   Model = claude-haiku-4-5-20251001
RULES     p95(Context usage) > 0.8
WINDOW    15 minutes

To cover a second model with the same alert, change the filter to in and list both. To give each model its own threshold, create one alert per model. An alert takes up to 10 filters with up to 50 values each, and up to 10 rules.

To tie the alert to one part of your agent, add a Service or Agent filter next to the Model filter. Filters combine, so the rule measures only calls that match all of them.

What you get

• An alert that fires only for calls to the model you chose, so a large-context model does not mask a small-window one.
• A threshold on how full the window gets, not on raw token counts, so the same 0.8 means the same thing for models with different window sizes.
• Firings that record the observed value of each rule against its threshold, so you can see which condition tripped.
• A coverage ratio on each firing, so you know how many calls had a known context window.
• A details page with the generated description, the editable rule, the attached channels and the firing history. See Alerts for how firings, channels and renotification work. A custom alert can be deleted, which removes its firing history after you confirm.

If you add other metrics to the same alert, remember the scope. With a Model filter, Duration runs per trace from the earliest matched span’s start to the latest matched span’s end, not across the whole run. Rules combine with AND only. For either-or behavior, create a second alert.

When something else fits

• If you want to know when whole runs get long or expensive rather than when one model’s window fills, filter by Service or Agent instead of Model, since those measure whole runs.
• If a model has no known context window, Context usage cannot cover it. Use an Input tokens rule on the same Model filter instead.
• If you want a standing alert for repeated failures rather than window pressure, the pre-defined alerts cover errors, loops and Context rot.

Related