How do I alert on error rate for one span kind?
In Rius, create a custom alert with an Error rate rule and add a Span kind filter, for example Span kind = tool. The filter narrows the spans before the rate is calculated, so the alert fires on the share of that kind’s spans that errored in the window. You compare the rate against a value between 0 and 1, such as 0.1 for 10%.
How it works
A custom alert is one or more rules that are combined with AND and evaluated over one shared rolling window. Filters are applied first, and the rules measure only the spans the filters match.
- The filter this needs. Span kind matches the kind of each span: LLM, tool, chain, retriever, and so on. With Span kind = tool, Error rate is the share of tool spans that errored in the window, not of every span in the run.
- Error rate is a ratio. Its only aggregation is rate, because a rate is already an aggregate. The value runs from 0 to 1, and the comparison operators are
<,<=,>,>=and==. - LLM error rate is the same kind of ratio, but over LLM calls. An LLM call is an inference call, a span the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count, even when they carry a model.
- Per-trace and per-call metrics. Numeric metrics such as Cost, tokens and Duration aggregate per trace, so
avg(Cost)is the average cost per trace in the window, not the workspace total. Time to first token, Context usage and LLM calls are measured per LLM call, so their aggregations are over calls. Event metrics such as Errors, Traces and LLM calls use count, or ucount for distinct traces. - Filters change what a metric covers. With a Span kind, Step, Model or Operation filter, Duration runs per trace from the earliest matched span’s start to the latest matched span’s end, not across the whole run. Cost sits on model-call spans, so a filter that selects only tool, chain or agent spans measures no cost.
Text filters offer =, ≠, in and not_in, and Rius suggests values from your workspace’s live data.
Set it up
- Open the custom alert builder in the Rius console. The Custom alerts page describes it in full.
- Add a filter: choose the Span kind field, the
=operator and the kind you care about, such as tool. To cover several kinds in one rate, useinand list them. Rius then gives you one combined rate, not one per kind. - Add a rule: choose the Error rate metric, the
>operator and a value between 0 and 1. - Set the window. It can be between 5 minutes and 24 hours, and all rules share it.
- Read the plain-language preview. It states exactly when the alert fires, and it becomes the alert’s description on its details page.
- Attach a channel and save.
The finished alert reads like this:
FILTERS Span kind = tool
RULES Error rate > 0.1
WINDOW 15 minutesThis fires when more than 10% of tool spans in the last 15 minutes errored.
To watch more than one kind separately, create one alert per kind. Rules combine with AND only, with no OR and no nesting, so a second alert is how you get either-or behavior. An alert takes up to 10 filters with up to 50 values each, and up to 10 rules.
If you want a low-traffic kind to alert on a raw number of failures instead of a share, use the Errors metric with the same filter and count, or ucount for distinct traces.
What you get
- An alert that tracks failures in one kind of step, such as tool calls or retrievals, without being diluted by the rest of the run.
- A threshold on a clear scale, 0 to 1, that you can read as a percentage.
- A description generated from your rule, and a details page that records the observed value of every rule against its threshold at each firing, so you can see which condition tripped.
- Routing through the channels you attach. See Alerts for how firings, channels and renotify work.
When something else fits
- If you want a failure rate across whole runs, filter by Service or Agent instead of Span kind, since those measure complete runs.
- If the only spans you care about are model calls, LLM error rate already limits itself to LLM calls, so you need no Span kind filter.
- If a small kind sees only a few spans per window, a single failure can swing a rate sharply. A count on Errors is steadier there.