How do I alert on LLM cost per customer?

Rius custom alerts cannot filter by customer, so you alert on cost per customer by giving each customer its own workspace and writing a cost rule in it. Alerts are configured per workspace, so each rule watches one customer and fires for that customer only. The rule measures cost per trace, not a customer’s total spend.

How it works

A custom alert filters the spans that feed its rules by service, agent, step, model, span kind, operation, session or instance. None of those fields is a customer or an end user. The session filter matches session.id, which scopes one conversation or task run, not everything a customer does.

A customer_id span attribute is stored on traces and you can inspect it, but it does not drive analytics, so it gives you no per-customer cost rollup and no per-customer alert.

The workspace is the boundary that does. Every trace belongs to exactly one workspace, set by the API key it was sent with. Cost rollups and alerts are per workspace, so a cost rule in a customer’s workspace evaluates only that customer’s traces. Acknowledging one customer’s alert never touches another’s.

Two details of the cost metric shape the rule you write:

• Cost is measured per trace, with avg, min, max, p50, p95 and p99 aggregations over a rolling window of 5 minutes to 24 hours. avg(Cost) is the average cost per trace in the window, not a total.
• A total-spend or budget threshold is not an alert metric. You watch totals on Usage and cost.

Set it up

1. Create a workspace for the customer from the workspace switcher (organization admins only), then create its API key in Settings → API keys inside that workspace. The number of workspaces is a plan limit, so check your plan before committing to one per customer. The full runbook is in Multi-tenant observability.

2. Send that customer’s traces with that customer’s key:

   export RIUS_API_KEY="ri_..."          # this customer's workspace key
   export RIUS_SERVICE_NAME="support-agent"  # the agent, same name across customers

   Use the same service name for the same agent across customers, so you can write the same rule in each workspace.

3. In that workspace, create a custom alert. This rule fires when the most expensive 5% of runs in the last hour cost more than $2.00 each:

   FILTERS   Service = support-agent
   RULES     p95(Cost) > $2.00
   WINDOW    1 hour

   Use max(Cost) to catch a single runaway run, or avg(Cost) to catch a general rise. Leave the filter off to cover every agent in the workspace.

4. Repeat step 3 for each customer’s workspace, adjusting the threshold to what that customer should cost. Rules within one alert combine with AND only, so write a second alert for either-or conditions.

5. Check which models are priced on Usage and cost. Spans whose model has no known price carry no cost, and a rule over mostly unpriced models can stay quiet. Firings show the share of the window that was priced.

What you get

• A cost alert that watches one customer and fires only for that customer, with its own threshold.
• Per-run protection: a prompt that grew or an agent stuck retrying shows up as a high p95 or max cost per trace.
• Each workspace’s monitoring view shows that customer’s spend, cost by model and token totals, so you can compare the alert against the totals.
• Costs are list-price estimates in US dollars from LiteLLM’s public model-price catalog, so expect your invoice to differ. For how a call is priced, see Track LLM and agent cost.
• With the MCP server, you can list your workspaces and fetch each one’s metrics with list_workspaces and workspace_metrics_overview, enough to script a weekly all-customers cost summary.

When something else fits

If your customers share one workspace, you cannot alert per customer. Tag end users with user.id and sort the Users page by cost to find your most expensive customers. If you need an alert on a customer’s total spend or a budget, Rius does not offer one today, so watch totals on Usage and cost instead.

Related