How do I monitor a research agent in production?

Rius shows a research agent run as one trace: a root span for the run, a span for each model call, and a span for each search or fetch step, with tokens, cost and errors on each. Wrap the run and each step with observe and give them the right kind, then watch the Traces, Agents and Alerts views. The general setup is in Monitor AI agents in production.

How it works

A run of a research agent becomes readable when each part of it has its own span:

• The run. Wrap the agent’s entry point with observe and pass kind=SpanKind.AGENT. This is the root span, so a run that ends in error counts as a failed trace.
• Each LLM call. With the instrumentation for your LLM library installed, every model call becomes a generation span with its model, token counts and cost. You send no cost value. Rius computes it as a list-price estimate in US dollars.
• Each tool or retrieval step. Wrap every search, fetch or read function with observe and kind=SpanKind.TOOL (kind: SpanKind.TOOL in TypeScript). A bare observe records a CHAIN span, not a tool span, so the kind matters for the Tool Loop alert below.

The Traces view then shows each run as a waterfall from the root span down to every generation and tool span, with token counts, cost and errors on the spans. A framework integration that emits its own tool and agent spans gives you the same shape without wrapping by hand.

Set it up

1. Install the SDK with the extra for your LLM library and call init with your API key and a service name. The monitoring guide covers account, key and a complete example.

   pip install "glassflow-rius[anthropic]" anthropic

2. Wrap the agent’s entry point with observe and kind=SpanKind.AGENT.

3. Wrap each search, fetch and read function with observe and kind=SpanKind.TOOL. Give every distinct step its own function, so each shows up as its own span in the waterfall.

4. Tag each release so you can compare versions later. Set these in the environment of the process:

   RIUS_MAIN_AGENT_VERSION=prompt-v12      # the agent definition: prompt, tools, policy
   RIUS_SERVICE_VERSION=2026.09.29-a41f3c  # the build or deploy

5. Shut down cleanly. End a short TypeScript script with await rius.shutdown(), because the TypeScript SDK does not flush spans when the process exits. If a deploy stops your process with SIGTERM, call shutdown() from your own signal handler so the instance ends as stopped and not gone.

6. Have a workspace admin create a Slack, email or webhook channel and attach it to each alert from that alert’s Delivers to section. Without a channel, a firing is only a row on the Alerts page.

What you get

Once the agent is live, these views answer the usual questions:

• Traces. Open a slow or failed run and read it step by step: which model call was expensive, which search errored, where the run stopped.

• Agents. The Agents page lists the agent with its errors and cost. Its page adds:

  • Drift charts for failed rate, cost per trace and time to first token.
  • Exceptions, which group errored spans by failure type, so you see one cause or several.
  • Context, where peak usage shows how close the largest call came to the model’s context window. This matters for runs that keep adding search results to the prompt.
  • Versions, which compares two tagged versions on failed rate, cost per trace, latency and tokens per trace. It warns when either side has fewer than 20 traces.

• Error rate against failed traces. A high error rate with few failed traces means the agent recovers from failing steps, for example by retrying a search. That costs time and money, but the result survives.

• Alerts. Three pre-defined detectors need no setup:

  • Consistent Error Pattern fires when the same error recurs across runs.
  • Swallowed Error fires when a run reports success while a step inside it that is not a tool call failed, such as a model call whose error your code caught. A failed search or fetch is a tool call, so look for it under the error rate and Exceptions instead.
  • Tool Loop fires when the same tool is called with identical arguments three or more times in a row within one trace. This catches an agent that keeps repeating one search. It needs TOOL spans, which step 3 gives you.

  Add custom alerts on workspace metrics, such as cost or duration, for thresholds of your own.

When something else fits

• Heartbeats show whether a process is alive, not whether the agent is stuck. A hung process that still sends heartbeats shows as running, so watch for runs that stay open too long in Traces.
• Rius does not score answer quality. Evals, datasets and experiments are not offered, so judging whether a report is correct needs a separate tool.
• Trace figures count only finished traces. A long run still in progress appears under Open traces and Recent traces until its last span ends.

Related