How do I alert on LLM cost per conversation?

Rius custom alerts can filter cost by session, but cost is measured per trace, so you alert on the cost of one turn, not the running total of a conversation. Use a Session filter with max(Cost) to catch an expensive turn in a conversation you name. To see what a whole conversation cost, filter Traces on session.id: the strip above the list sums the cost of that conversation’s traces. Over MCP, agent_traces_summary with a session_id filter returns the same total.

How it works

A custom alert is one or more rules, combined with AND, over a rolling window of 5 minutes to 24 hours. Optional filters narrow which spans feed the rules before anything is aggregated, and one of the filters is Session, which matches session.id. A session is your id for one conversation, and each turn of a chat is usually its own trace.

Cost is a numeric metric with the aggregations avg, min, max, p50, p95 and p99. These work per trace: avg(Cost) is the average cost of a trace in the window, not a total. Three consequences follow:

• With a Session filter, the rule reads the traces of that session, so a threshold fires when one turn costs too much.
• There is no sum aggregation, so a rule cannot add up the turns of a conversation and compare the total to a budget.
• A total-spend or budget threshold is not an alert metric today.

The Session filter takes values you choose, such as one session id. It does not run a separate check for every conversation. If your app puts a whole conversation in a single trace, per-trace cost is the conversation’s cost, and the rule below covers it fully.

Costs are list-price estimates in US dollars, and a call whose model has no known price carries no cost. See Track LLM and agent cost for how calls are priced.

Set it up

1. Set session.id so traces carry your conversation id. Use rius.session in Python, withSession in TypeScript, or RIUS_SESSION_ID for a whole process. Without it, every trace is its own session, and a Session filter has nothing useful to match. Concepts explains how sessions group traces.
2. Open the custom alert builder and add a Session filter. Values are suggested from your workspace’s live data.
3. Add a cost rule with max, or p95 if you want to ignore one-off spikes, and choose a window. Here is a rule for one conversation:

FILTERS   Session = chat-881
RULES     max(Cost) > $1.00
WINDOW    1 hour

4. Narrow it further if you need to. Add a Service, Agent or Model filter, so the rule covers one agent or one model within that conversation.
5. Attach a channel. Firings show the share of the window that was priced, so you can tell a high reading from a partly priced one. A rule over mostly unpriced models can stay quiet without firing.

For a rule that is not tied to one conversation, drop the Session filter and watch every turn of an agent, as in p95(Cost) > $2.00 with a Service filter. The custom alerts page has the full rule grammar.

What you get

• An alert when a single turn in a conversation you name crosses a cost threshold.
• An alert on expensive turns across a whole service or agent, which is what catches a prompt that grew or an agent stuck retrying.
• Per-firing detail: the observed value of each rule against its threshold, plus the priced share of the window.
• Cost for the whole conversation to read, not to alert on. Cost per trace shows in the trace list and trace header, and total cost per end user shows on the Users page, which you can sort by cost. The workspace total for any period is on the Usage and cost page. See Track LLM and agent cost for where each of these lives.

When something else fits

• If you need a threshold on the combined cost of every turn in a conversation, a custom alert cannot do it. On Traces, filter on session.id and the strip above the list sums the cost of that conversation’s traces, or watch spend by end user on the Users page.
• If you need a budget for the whole workspace, watch it on Usage and cost, since total spend is not an alert metric.
• If one workspace per customer fits your setup, the workspace total is that customer’s cost.

Related