How do I get alerted when my agent is slow to respond?

Rius alerts you when your agent is slow to respond through a custom alert on the Duration metric, or on Time to first token for streamed replies. You set a percentile threshold over a rolling window, attach a channel such as Slack, email or a webhook, and Rius opens a firing with the affected traces when the threshold is crossed. The firing page links straight to the slow runs.

How it works

The pre-defined alerts cover recurring errors, swallowed errors and tool loops, so slowness is a custom alert. A custom alert is one or more rules on a workspace metric, evaluated over one rolling window of 5 minutes to 24 hours. Two metrics fit this question:

• Duration, in milliseconds. It measures how long a whole run takes. The aggregations are avg, min, max, p50, p95 and p99, and each is computed over traces, so p95(Duration) is the 95th percentile of run time across the traces in the window.
• Time to first token, in milliseconds. It measures how long a streamed model call waits before its first chunk arrives. It is measured per model call, not per trace, and it exists only on streamed calls whose instrumentation records it. Firings on this metric show the coverage ratio, so you can tell a real reading from a partly covered one.

Use a percentile such as p95 when you care about the slow tail, and avg when you care about the typical run. You can add filters for service, agent, step, model, span kind, operation, session or instance, so a slow checkout agent does not hide behind a fast one. Several rules in one alert combine with AND only. For either-or, create a second alert.

Rules are evaluated every minute while no firing is open and every five minutes while one is, so timings are approximate to that cadence. The main Alerts page covers firings, acknowledgment and the workspace-wide Alerting switch.

Set it up

1. Pick a threshold from real numbers. Open Home, which is described under Usage and cost, and read the p50 and p95 latency and the time to first token charts for the period. Latency percentiles there count finished traces only, so a run still in progress does not stretch them.
2. On the Alerts page, create a custom alert. Add an optional filter, then a rule on Duration or Time to first token. A rule for a service that should answer within 30 seconds at the 95th percentile looks like this:

FILTERS   Service = checkout-agent
RULES     p95(Duration) > 30000
WINDOW    15 minutes

3. Set the window. A short window reacts faster, and a longer one smooths out single slow runs. The builder shows a plain-language preview of exactly when the alert fires, and that sentence becomes the alert’s description.
4. Attach a distribution channel (webhook, email or Slack) so the firing reaches you.
5. Set Renotify to repeat the notification every 5 to 1440 minutes until someone acknowledges the firing. The default is off, which sends one notification per firing.
6. Optionally turn on the alert’s Root-cause analysis switch, so every firing is handed to the analysis agent when the workspace has root-cause analysis enabled.

Configuring alerts is an admin action. Members can see firings and acknowledge them.

What you get

When the condition holds, Rius opens a firing and sends it to every channel attached to the alert. From there:

• The firing page shows when it opened, when the condition was last seen and the affected traces, each linked to the trace view. It also lists the delivery outcome per channel.
• The observed value of each rule is recorded against its threshold, so you see how slow the runs were, not only that the alert tripped.
• View all in Traces opens the trace list filtered to the firing’s time window, so you can open the slowest runs and look at the span timings.
• A root-cause report, when analysis is on, says what failed, since when and the likely cause.
• Acknowledgment records who saw the firing and stops further notifications. The firing resolves on its own after the condition has stopped matching for two hours. If slowness returns later, it opens a new firing.

Home stays useful alongside the alert. Its latency and time to first token charts show whether the slowness is a spike or a trend, and a per-model time to first token p95 shows whether one model is the cause.

A rule such as Traces count < 1 over a generous window also fires if ingestion stops entirely, which catches an agent that has gone quiet rather than slow.

When something else fits

If your concern is runs that fail rather than run long, the pre-defined Consistent Error Pattern or Swallowed Error alerts fit better than a latency rule. If you only want to watch speed trends without being notified, the latency charts on Home give you that without any alert.

Related