Which of my AI agents is getting worse?

Rius shows which agent is getting worse on the Agents page: compare errors, cost and time to first token per agent, then open one to see drift and versions. Every agent sending traces is listed, with no registration. Rows with problems come first. Open an agent to see whether the failed rate, cost per trace or time to first token is rising, and whether a new version caused it. This is one part of Monitor AI agents in production.

How it works

The Agents page finds agents from your traces and heartbeats. Each row shows one agent over the time window you select: its instances, open traces, traces, errors, time to first token and cost, plus when it was last seen. Rows with problems come first, so the worst agent is usually near the top. Cost is a list-price estimate in US dollars, not a billed amount.

The trace figures count only finished traces. A run appears in them once every span has ended, and a long session still in progress never skews a latency percentile.

When you open an agent, the page reads from top to bottom:

• Summary tiles: Traces, Failed traces, Error rate, TTFT p50 (median time to first token) and Cost with average cost per trace.
• Drift: three charts for TTFT p50, cost per trace and failed rate, with one point per minute, hour or UTC day depending on the window. Each chart lists its peak and low.
• Versions: which versions ran in the window, with a Compare view that puts a base version next to another.
• Exceptions: errored spans grouped by failure type, so you can tell one cause from several.

Error rate and Failed traces measure different things. A failed trace is a run whose root span ended in error. Error rate also counts runs where a span errored but the agent recovered. A high error rate with few failed traces means the agent recovers through retries or fallbacks, which costs time and money but keeps the result intact.

Set it up

1. Send traces with the Rius SDK. Heartbeats are on by default, so your agent appears on the Agents page. The Agents page covers the full setup.
2. Tag each release so Rius can compare versions. Set the agent definition version and the build version in the environment of the process:

RIUS_MAIN_AGENT_VERSION=prompt-v12      # the agent definition: prompt, tools, policy
RIUS_SERVICE_VERSION=2026.09.29-a41f3c  # the build or deploy

3. Change the version value every time you change the prompt, tools, model or build. Without a version, the page tells you which variable to set.
4. Open Agents and choose a time window: 15m, 1h, 1d, 7d or 30d. Scan the Errors and Cost columns and the time to first token for the agent that stands out.
5. Click the row. On the agent page, check Drift for a rising failed rate, cost per trace or TTFT p50. Then open Exceptions to see whether one cause or several explain the errors.
6. If two or more versions ran in the window, use Compare in Versions. It shows traces, failed rate, cost per trace, latency and time to first token (p50), and tokens per trace side by side, with the change. It warns you when either side has fewer than 20 traces, because the difference can then be noise.
7. Click View all in Traces from Recent traces or Errored traces to open the filtered trace list for that agent and read the failing runs.

To ask the same questions from an agent, the Rius MCP server offers list_agents, agent_versions, compare_agent_versions and agent_error_groups.

What you get

• One list that shows which agent has the most errors, the highest cost or the slowest time to first token over the window you pick.
• A drift view that shows when the failed rate, cost per trace or TTFT p50 started to move, so you can match it to a deploy.
• A version comparison that tells you whether the last prompt, model or build helped or hurt, from the day it shipped.
• A grouped view of errors by failure type, so you know whether to fix one cause or several.
• A clear split between runs that failed and runs that recovered, so retries and fallbacks that quietly add cost do not hide.
• Instance status (running, ready, stale, gone, stopped), so you can tell a crash from a clean shutdown. An instance that ends as gone usually points to a crash, an out-of-memory kill or a deploy that sent SIGTERM.

When something else fits

• Rius measures failures, cost and latency. It does not offer scores, feedback or evals, so an agent that returns weaker answers without erroring will not show up as worse.
• Heartbeats show that a process is alive, not that it is making progress. A hung process that still sends heartbeats shows as running, so check open traces and durations as well.

Related