How do I alert on context window usage for one step?

In Rius, you alert on context window usage for one step by creating a custom alert with a Step filter and a rule on Context usage, such as p95(Context usage) > 0.8. The Step filter must match the name of the model-call span you want to watch. The alert then fires when the fullest calls from that step fill more than 80% of their model’s context window.

How it works

A custom alert applies its filters first and measures only the spans that pass. Each filter matches a span’s own value. The Step filter matches the span name, exactly.

Context usage is the share of the model’s context window that one LLM call filled. Rius takes the call’s input tokens, prompt cache included, and divides them by the model’s window. You compare it against a value between 0 and 1.

Context usage exists only on LLM calls, which are the spans the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count, even when they carry a model. So the Step you filter on has to be the span for the model call itself. If you name a step that is a chain, agent or tool span, the alert has no model calls to measure.

How a metric aggregates matters here:

• Most numeric metrics, such as Cost and Duration, aggregate per trace. avg(Cost) is the average cost per trace in the window.
• Context usage, Time to first token and LLM calls are the exception. They are measured per LLM call, so avg, p95 and the other aggregations run over calls, not traces.
• The aggregations offered for Context usage are avg, min, max, p50, p95 and p99.

A call whose model has no known context window has no context usage. Rius leaves it out and never counts it as 0.

Set it up

1. Open the custom alert builder. For the shared steps of creating an alert, see Custom alerts.
2. Add a Step filter with = and the exact span name of the model call. Values are suggested from your workspace’s live data, so pick the name from the list rather than typing it. Use in if the step runs under more than one name.
3. Optionally add a Span kind filter set to LLM. This keeps the alert on model calls only.
4. Add a rule on Context usage. Choose an aggregation such as p95 and a threshold between 0 and 1. A value of 0.8 means 80%.
5. Set the window, from 5 minutes to 24 hours.
6. Attach a channel (webhook, email or Slack) so the firing reaches you.

The builder shows a plain-language preview of when the alert fires. The alert is built like this, where summarize_step stands for your own span name:

FILTERS   Step = summarize_step
   AND    Span kind = LLM
RULES     p95(Context usage) > 0.8
WINDOW    15 minutes

Keep the alert to context rules. All rules in one alert share the same filters and window. If you add a Duration rule next to a Step filter, it runs from the earliest matched span’s start to the latest matched span’s end, not across the whole run. Create a second alert for whole-run metrics, since rules combine with AND only and there is no OR.

The Step filter matches names exactly. If you upgrade an agent to the 1.0.0 SDKs, unnamed model, tool, agent and retrieval spans are named after their operation and target, such as execute_tool search_orders. Update the filter to the new name after the upgrade.

What you get

• An alert scoped to the model calls of one step, so a prompt that keeps growing is caught where it happens and not averaged away across the whole agent.
• A threshold you read as a percentage of each model’s own window, so the same rule works across models with different window sizes.
• A firing that records the observed value of each rule against its threshold, so you can see which condition tripped.
• A coverage ratio on each firing. Context usage exists only for calls whose model has a known window, and the ratio tells you whether a reading is complete or partly covered. A rule held under its threshold by low coverage stays quiet, so a quiet alert does not prove every call was measured.
• A generated description on the alert’s details page, taken from the builder’s preview sentence.

When something else fits

• To watch context across an entire agent, filter on Service or Agent instead of Step. Those filters cover every span that carries the value.
• To alert on spend inside one step, note that no filter measures the cost of the model calls inside a step. Filter Model, or Span kind set to LLM, to alert on model spend. See Alerts for the other alert types.
• If you want either-or conditions, such as context above 80% or errors above a count, create a separate alert for each condition.

Related