Skip to Content

My AI agent hits errors from the model provider. How do I find the cause?

Rius shows provider errors on the span for the failing model call, so you can go from the agent page to the exact call and read its exception. Open the agent’s Exceptions section to see which failures repeat, open an errored trace, then read the failed LLM span. The exception, traceback, model, tokens and context usage on that span point at the cause. This is the same path described in Monitor AI agents in production.

How it works

Rius derives trace status from the spans, not from what your agent reports. A span with an OpenTelemetry ERROR status marks its trace failed. If the run’s root span finished cleanly after retries or a fallback, the trace is badged recovered instead. The errors still count either way.

Three places show the symptom:

  • Agent page. Agents compares Error rate with Failed traces. A high error rate with few failed traces means the agent recovers from provider errors, which costs time and money without breaking results. Exceptions groups errored spans by failure type, such as llm request failed, with when each last happened and how many traces and spans it hit.
  • Trace waterfall. Exploring traces highlights failed spans along the path from the root to the error. For an error span, the detail panel leads with the exception and traceback.
  • LLM span detail. The Model section shows the provider, requested and response model, input and output tokens, and cost. When the model’s context window is known, it also shows how much of it the call used.

Set it up

  1. Start on the agent page. Open Agents, pick the agent, and set the time window to when the errors began. Read Exceptions first. One group means one cause. Several groups mean several. Then check Drift, where the failed rate chart shows when errors started, and Errored traces, which lists the six newest traces with an errored span.

  2. Open one errored trace. In Recent traces, failed and recovered rows are marked by state. In the waterfall, follow the highlighted path down to the failed span. If you see a row like fetch_article · 3 attempts · recovered, the call was retried. Expand it to compare the attempts.

  3. Read the failed LLM span. Check these in order:

    • The exception and traceback at the top of the detail panel.
    • The Model section: is it the model you meant to call? Compare the requested and response model.
    • Token counts and context usage. A call that was very large for its model’s window is a lead. The context growth strip above the tree plots input tokens per call, so a runaway prompt or unbounded history shows there first.
    • Input / Output, with Since previous llm call to see what the last step added to the prompt. If your SDK’s privacy controls strip content, the tab says so and the rest of the span still works.
    • Raw, which shows every attribute as ingested. Each attribute has a Filter traces by this menu.

  4. Find out if it is one model or all of them. On the Spans tab, choose the LLM kind in the All kinds menu and filter by status. Then add a filter on the model attribute from the Raw menu. If the failures cluster on one model, the cause is likely on that model’s side. If they spread across models, look at your prompts or your own code.

  5. Tag releases so you can tell if a change caused it. Set these two variables, then use Versions and Compare on the agent page. Compare warns when either side has fewer than 20 traces. The full setup is in Installation and configuration.

RIUS_MAIN_AGENT_VERSION=prompt-v12      # the agent definition: prompt, tools, policy
RIUS_SERVICE_VERSION=2026.09.29-a41f3c  # the build or deploy
  1. Get told next time. Build a custom alert on the LLM error rate metric, which is the share of LLM calls that errored in the window, compared against a value between 0 and 1. Scope it with a Model or Service filter. See Custom alerts.

What you get

  • A short path from the symptom to the call: agent page, errored trace, failed span.
  • Errors grouped by failure type, so you can see whether you chase one cause or several.
  • A clear split between runs that failed and runs that recovered, so you know whether users saw the error.
  • Model, token and context details on each failed call, plus the prompt and response when content is captured.
  • A drift chart and version comparison that show whether a release, a prompt change or a model change lines up with the errors.
  • An alert on the LLM error rate, so the next spike reaches you before you go looking.

When something else fits

  • If the errors are in your own code rather than model calls, the same path works, but filter the Spans tab to the Tool or Chain kind instead of LLM.
  • If the agent hangs rather than errors, instance heartbeats show only that the process is alive, so look for traces that stay running… (these appear only when partial spans are enabled).

Still stuck? Get help

Last updated on
Quickstart