How do I get alerted when my agent runs out of context window?

Rius has no pre-defined alert for a full context window, so you create a custom alert on the Context usage metric. A rule such as p95(Context usage) > 0.8 fires when the fullest 5% of LLM calls in the window fill more than 80% of their model’s context. Each firing is recorded in Rius and delivered to the channels you attach, and you can open it to see the value that tripped the rule and the traces behind it.

How it works

Context usage is the share of the model’s context window that one LLM call filled. Rius divides the call’s input tokens, prompt cache included, by the model’s window. You compare it against a value between 0 and 1, and the builder shows it as a percentage.

• Measured per call. Context usage is measured on each LLM call, so the aggregations (avg, min, max, p50, p95, p99) run over calls, not traces. Agent, tool, retrieval and embedding spans never count, even when they carry a model.
• Choosing the aggregation. Use max to catch any single call that gets close to the limit. Use p95 or p99 to catch a run where the fullest calls keep growing, without reacting to one outlier.
• Models without a known window. A call whose model has no known context window has no context usage. It is left out of the aggregation and never counted as 0.
• Coverage. Each firing shows a coverage ratio, so you can tell a real reading from one based on only some of your calls. A rule held under its threshold by low coverage stays quiet.
• Evaluation cadence. Rius checks the rule every minute while it has no open firing, and every five minutes while one is open.

The pre-defined alerts are Consistent Error Pattern, Swallowed Error and Tool Loop. None of them watches context, which is why the custom builder is the right place. See Custom alerts for the full rule grammar.

Set it up

Configuring alerts is an admin action. Members can see alerts and acknowledge firings.

1. Open the Alerts page and create a custom alert. The builder previews in plain language when the alert fires, and that sentence becomes the alert’s description.
2. Optionally add a filter so the rule only sees the agent you care about, for example Service, Model or Step. Filters are applied before aggregation, and the builder suggests values from your live data.
3. Add one rule on Context usage, with an aggregation and a threshold between 0 and 1.
4. Set the window. It can be anywhere from 5 minutes to 24 hours and is shared by all rules in the alert.
5. Attach a distribution channel (webhook, email or Slack) so the firing reaches you.
6. Optionally set Renotify to repeat the notification every 5 to 1440 minutes until someone acknowledges the firing. The default, off, sends one notification per firing.
7. Save the alert.

This is the rule from the builder, with a service filter, as it reads in the grammar:

FILTERS   Service = checkout-agent
RULES     p95(Context usage) > 0.8
WINDOW    15 minutes

Rules in one alert combine with AND only. Adding count(Errors) > 10 makes the alert fire only when calls are near the limit and errors rise together. To be told about errors on their own, create a separate alert.

What you get

When the condition matches, a firing opens and the Alerts entry in the sidebar shows a count of firings nobody has acknowledged. Open the firing to see where to look next:

• The observed value against your threshold. Each rule’s value is recorded, so you see how full the context actually got.
• The coverage ratio. It shows how much of the data the reading rests on.
• The affected traces. Each links to the trace view, and View all in Traces opens the trace list filtered to the firing’s time window.
• The delivery list. One row per channel, with the outcome of each attempt.
• A root-cause report. If root-cause analysis is on for the alert and the workspace, the page carries the analysis agent’s report.

Acknowledge the firing to record who saw it and stop further notifications. It resolves on its own once the condition has stopped matching for two hours, and a later recurrence opens a new firing. If you change the rule, saving resolves its open firings and the rule is evaluated afresh. The Alerts page covers firings, renotify, the workspace-wide alerting switch and rule health in full.

When something else fits

If you care about raw prompt size instead of the share of the window used, a custom alert on Input tokens is the better fit. It is also the better fit when your model has no known context window, because those calls carry no context usage and are left out of the Context usage rule. If you only want to know that traces have stopped arriving, a Traces count < 1 rule over a generous window does that.

Related