How do I alert on cost per trace for one span kind?

Rius lets you alert on cost per trace for one span kind by adding a Span kind filter to a custom alert with a rule such as avg(Cost) > 0.50. The filter limits which spans feed the rule before anything is aggregated. Cost aggregates per trace, so avg(Cost) is the average cost per trace in the window, not the workspace total.

How it works

A custom alert is a set of optional filters, one or more rules and one rolling window. Filters run first and select which spans feed the rules. Aggregation happens after that.

• The filter you need. The Span kind field matches a span’s kind: LLM, tool, chain, retriever and so on. It offers =, ≠, in and not_in, and suggests values from your workspace’s live data. Use in to cover several kinds in one alert. Cost lives on model-call spans, such as LLM and embedding spans. A filter on a kind that makes no model call, such as tool or chain, selects spans with no cost, so the rule has nothing to measure.
• The metric. Cost is a numeric metric in USD. It offers avg, min, max, p50, p95 and p99.
• Per-trace aggregation. Numeric aggregations use per-trace semantics. avg(Cost) > 0.50 means the average cost per trace in the window is above $0.50.
• Per-call metrics. Time to first token, Context usage and LLM calls are measured per LLM call, so their aggregations run over calls, not traces. Cost is not in that group.
• Combining rules. Rules combine with AND only. For either-or behavior, create a second alert.

Cost is only as complete as your pricing coverage. A span whose model has no known price carries no cost. Firings show the coverage ratio, but a rule held under its threshold by low coverage stays quiet.

Set it up

1. Open the custom alert builder from Custom alerts and start a new alert.
2. Add a filter on Span kind. Choose = and pick the kind, for example LLM. Add other filters, such as Service or Model, if you want to narrow the alert further.
3. Add a rule on Cost. Pick an aggregation (avg gives the average cost per trace, p95 gives the cost of your more expensive traces), an operator from <, <=, >, >= and ==, and a value in USD.
4. Set the window. It is shared by all rules and runs from 5 minutes to 24 hours.
5. Read the plain-language preview. It states exactly when the alert fires, and it becomes the alert’s description.
6. Attach your channels and, if you want repeat notices, set the Renotify policy.

The alert you build looks like this:

FILTERS   Span kind = LLM
RULES     avg(Cost) > $0.50
WINDOW    15 minutes

An alert takes up to 10 filters with up to 50 values each, and up to 10 rules.

Before you rely on the alert, check which models are priced on Usage and cost. Costs are list-price estimates in US dollars, and a model with no known price adds no cost to the average.

What you get

• An alert that tracks the cost per trace of one span kind, such as LLM spans, without counting spans of other kinds.
• A preview sentence that tells you what will trigger the alert before you save it.
• Firings that record the observed value of every rule against its threshold, so you can see which condition tripped.
• A coverage ratio on each firing, so you can tell a real reading from a partly priced one.
• A details page for the alert where you can edit the rule, attach channels and delete the alert when you no longer need it.

For how alerts fire, notify and renotify, see Alerts.

When something else fits

• If you want a per-call reading rather than a per-trace one, use Time to first token or Context usage, which are measured per LLM call.
• If you need either-or logic, for example one cost threshold for LLM spans and another for embedding spans, create one alert for each, because rules combine with AND only.
• If you only want to look at cost rather than be paged on it, the summary on Usage and cost shows the same metric without a trigger.

Related