Which customers will a change to my AI agent affect?

Rius shows which customers an agent serves: the Agents column on the Users page lists the agents that served each end user, so you can check it before a rollout. Before you ship a new prompt or model to one agent, scan that column for the agent you are changing. The rows that include it are the customers who will notice. Rius also shows each user’s failures, cost and time to first token, so you can see who has the most exposure.

How it works

Rius ties each trace to an end user through the user.id attribute. It also accepts enduser.id or enduser.pseudo.id and takes the first one that is set. A trace with no end user on any span counts as anonymous.

The Users page counts finished traces per user and lists the agents that served each one. A user’s own page shows up to three agents that handled their traces in the window, most first. Per-agent counts for one user are not shown yet, so you see which agents served a user, not how many requests each handled.

The page also has a time window of 15m, 1h, 1d, 7d (the default) or 30d. Pick a window that matches how often your customers use the agent. A customer who calls it once a month will not show in a 1d view.

Set it up

1. Attribute requests to end users. The SDKs set user.id on every span of a request that runs inside a user scope. In Python that scope is rius.user, in TypeScript it is withUser. Set it on every code path, including the root span, and prefer an opaque user ID to an email. Initialize the SDK once with the usual setup:

import rius
 
client = rius.init(
    api_key="ri_...",
    service_name="my-agent",
)

2. Verify with a test user. Run one request inside a user scope with an ID such as test-user-1. Open Users, type that ID, click Find and open the row. Once the trace has finished, the page shows 1 trace, your agent under Agents that served this user, and the trace under Recent traces.
3. Check how much traffic you can attribute. The line above the list reads like “400 of 471 traces carry an end user”. Everything else sits in the anonymous row, which always comes first. If that row is large, some code paths run outside a user scope, and customers on those paths will not appear against your agent. Fix that before you rely on the list for a rollout.
4. Find your exposure. On Users, choose a window and sort by traces, failed traces or cost. Read the Agents column and note every user whose row includes the agent you are about to change. The list shows the top 50 users for the chosen sort. To reach anyone else, search by their exact user ID.
5. Open the users who matter most, such as your highest-cost or most failure-prone customers. Under Agents that served this user, click an agent to open its page.
6. Compare each user’s tiles with the agent’s own numbers. If a user’s error rate or cost per trace is well above the agent’s, the problem is probably specific to that user, and a change may hit them harder.
7. After the rollout, open Traces → on a user’s page, or filter the trace list by user. The same filter is the user_id field in the filter builder and the user_id argument of the trace tools on the MCP server. The MCP tools list_users and user_activity_summary also take it.

What you get

• A list of the customers an agent serves, read from the Agents column before you change a prompt or model.
• For each user, tiles for Traces, Failed traces, Error rate, Cost, Sessions and TTFT p50, so you know who is already struggling or already expensive.
• A way to see whether failures sit with a few users. Sort by Failed and look at what they share, such as account settings, data size or language, before you change anything that affects everyone.
• A path from a customer’s complaint to the failing span. Search their exact ID, open their page, check Failed traces and open the trace from the time they wrote in.
• The 8 most recent traces and the 6 latest failed traces for each user, to compare behavior after the change.
• An optional written summary of a user’s experience. When root-cause analysis is on for the workspace, an admin clicks Summarise this user. Each summary counts as one analysis against the same budget as root-cause analysis on alerts.

For the wider picture of watching agents after a change, see Monitor AI agents in production.

When something else fits

• If the anonymous row holds much of your traffic, the Users list cannot tell you who a change will affect. Open that row to see the traces with no end user, and add a user scope to those code paths first.
• If you need exact per-agent request counts for one customer, use the trace list filtered to that user, since the user’s page shows which agents served them but not counts.
• If a customer only appears in a window you have not selected, widen it to 30d before you conclude they are unaffected.

Related