How do I alert on context window usage for one service?

Rius lets you alert on context window usage for one service with a custom alert that has a Service filter and a rule on the Context usage metric, such as p95(Context usage) > 0.8. The filter limits the alert to spans from that service before anything is aggregated. Context usage is measured per LLM call, so the rule looks at the fullest calls in the window, not at whole traces.

How it works

A custom alert is one or more rules combined with AND, evaluated over one shared rolling window. Optional filters pick which spans feed the rules, and they apply before aggregation.

The filter this needs. Use the Service filter. It matches service.name from your instrumentation, and the builder suggests values from your workspace’s live data. Set it to = with your service name. Use in if you want several services in one alert, or not_in to exclude some.

What Context usage measures. It is the share of the model’s context window that one LLM call filled. Rius divides the call’s input tokens, prompt cache included, by the model’s window. You compare it against a value between 0 and 1, so 0.8 means 80%. An LLM call here is an inference call, a span the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count, even when they carry a model.

How per-trace and per-call metrics aggregate. Most numeric metrics, such as Cost, tokens and Duration, are computed per trace. avg(Cost) is the average cost per trace in the window, not the workspace total. Three metrics are the exception and are measured per LLM call: Time to first token, Context usage and LLM calls. Their aggregations run over calls. For Context usage, the available aggregations are avg, min, max, p50, p95 and p99.

So p95(Context usage) > 0.8 fires when the fullest 5% of that service’s calls in the window use more than 80% of their model’s context.

Coverage. A call whose model has no known context window has no context usage. Rius leaves it out and never counts it as 0. Firings show the coverage ratio, so you can tell a full reading from a partly covered one. A rule held under its threshold by low coverage stays quiet.

Set it up

1. Open the custom alert builder and add a filter: Service = the name of your service, for example checkout-agent.
2. Add a rule on Context usage. Pick an aggregation that matches what you want to catch. p95 or max catch the fullest calls, while avg smooths them out. Choose > and a value between 0 and 1.
3. Set the window. It is shared by all rules and can be between 5 minutes and 24 hours.
4. Read the plain-language preview. It states exactly when the alert fires, and it becomes the alert’s description on its details page.
5. Attach channels and set the Renotify policy, off or every N minutes, then save.

The alert reads like this:

FILTERS   Service = checkout-agent
RULES     p95(Context usage) > 0.8
WINDOW    15 minutes

A few things to know while you set it up:

• Rules combine with AND only. You could add a second rule, such as a count of LLM calls above a minimum, so the alert ignores quiet windows with very few calls. For either-or behavior, create a second alert.
• Keep the filter on Service alone for this alert. Step, Model, Span kind and Operation filters change what a metric covers, so add one only if you want to narrow the alert to a model or a step.
• An alert takes up to 10 filters with up to 50 values each, and up to 10 rules.

The Alerts page covers how firings work in general.

What you get

• An alert scoped to one service, so a long-context workload elsewhere in your workspace does not trigger it.
• A threshold on the calls that matter. With a high percentile or max, a few calls close to the model’s limit trip the alert even if the average looks fine.
• A details page with the generated description, the editable rule, attached channels and firings. Each firing records the observed value of every rule against its threshold, so you can see which condition tripped.
• Coverage on each firing, so you know how many of the service’s calls had a known context window.
• Control over noise through the window length and the Renotify policy.
• A rule you can delete when you no longer need it. Deleting removes its firing history after an explicit confirmation.

When something else fits

• If you want to catch runaway context across every service, drop the Service filter, or use in to list several services in one alert.
• If you care about how much a whole run costs or how long it takes, use a per-trace metric such as Cost or Duration with the Service filter. Those aggregate per trace, not per call.
• If your service uses models with no known context window, Context usage has nothing to measure for those calls, so a different metric is a better signal.

Related