How do I get an alert on my agent’s error rate?

Rius lets you build a custom alert on the Error rate metric, scoped to your agent with a filter, that fires when the share of errored spans in a rolling window crosses a threshold you choose. Error rate uses the rate aggregation and you compare it against a value between 0 and 1, so 0.05 means 5% of spans errored. Each firing goes to the channels attached to the alert and records the observed value against your threshold. Setting up alerts is an admin action. Members can see firings and acknowledge them.

How it works

A custom alert is one or more rules combined with AND, evaluated over one shared rolling window, with optional filters applied before the rules run. The Alerts page holds all of them. The custom alerts page covers the full builder.

For error rate, these points matter:

• Aggregation. Error rate is a ratio metric. The only aggregation is rate, because a rate is already an aggregate. You do not pick avg or p95 for it.
• What it measures. Error rate is the share of spans that errored in the window. LLM error rate is the same share, counted over LLM calls only. Use LLM error rate when you care about failing model calls and not failing tools or chain steps.
• Scope. Filter on Service, Agent, Step, Model, Span kind, Operation, Session or Instance. Each filter matches a span’s own value, so an Agent filter selects every span that carries that agent, not only root spans.
• Operators. Use <, <=, >, >= or ==.
• Window. Anything from 5 minutes to 24 hours.
• Evaluation. Rius checks the rule every minute while it has no open firing, and every five minutes while one is open.

The builder shows a plain-language preview of when the alert fires, and that sentence becomes the alert’s description.

Set it up

1. Open the Alerts page and create a custom alert. You need an admin role.
2. Add a filter for the agent you care about, for example Service = checkout-agent. Without a filter the rule covers every span in the workspace.
3. Add a rule on Error rate with the rate aggregation, the > operator and a value such as 0.05.
4. Set the window. A 15 minute window is a reasonable start: short enough to catch a bad deploy, long enough that one failed call does not trip it. Widen it if the alert is noisy.
5. Optionally add a second rule with AND so a handful of spans cannot trip the alert on their own, such as count(Errors) > 10. Both rules must hold for the alert to fire.
6. Attach one or more distribution channels (webhook, email or Slack) and set Renotify to off or every N minutes.
7. Save. Saving a change resolves the alert’s open firings and starts the rule afresh.

The finished rule has this shape:

FILTERS   Service = checkout-agent
RULES     rate(Error rate) > 0.05
   AND    count(Errors) > 10
WINDOW    15 minutes

The threshold and window above are examples. Pick values that match your agent’s normal error level. Rules combine with AND only. If you want one alert for either condition, create a second alert.

What you get

• A notification on every attached channel. While the alert is enabled, each firing is delivered to every channel attached to it. The firing’s page lists one delivery row per channel with the outcome of each attempt.
• The observed value against your threshold. Firings record the observed value of every rule, so you can see which condition tripped.
• Affected traces. The firing page links the traces it kept, and View all in Traces opens the trace list filtered to the firing’s time window.
• Reminders on your terms. With Renotify off you get one notification per firing. With it set to every N minutes, from 5 to 1440, the notification repeats until someone acknowledges the firing or it resolves.
• A clear lifecycle. The firing stays open while the condition keeps matching. It resolves on its own after two hours without a match. If the condition returns later, that is a new unacknowledged firing. Acknowledging records who saw it and stops further notifications for it, but does not close it.
• Optional root-cause analysis. Turn on the alert’s Root-cause analysis switch and, when the workspace has analysis enabled, each firing is handed to the analysis agent, which writes a report on the firing page.

If a rule fails to evaluate repeatedly, its chip changes to Failing and then Suspended. Save any change to the alert to resume it.

When something else fits

• Use LLM error rate instead of Error rate when only failing model calls matter to you.
• If you need a count and not a share, such as more than 10 errors in the window, use the Errors metric with count, or ucount for distinct traces.
• To be told when ingestion stops entirely, a Traces count < 1 rule over a generous window does that. An error rate rule cannot, because it has no spans to measure.

Related