Skip to Content

My OpenAI Agents SDK agent fails only for some users. Why?

Rius shows you which users fail and which step fails for them, so you can see what those users have in common. Sort the Users page by failed traces, open an affected user’s failed traces, and read the failing span in the waterfall. For an OpenAI Agents SDK agent that span is a model call (an LLM generation named Response) or a tool call you wrapped with observe.

How it works

The OpenAI Agents SDK integration records three kinds of spans for each run:

  • A root AGENT span from your own observe wrapper around the entrypoint.
  • An LLM generation for every model call, with model, tokens and cost.
  • A TOOL span for every function tool whose body you wrapped with observe.

Each span carries a user.id when the request runs inside a user scope (rius.user in Python). Rius uses that id to group traces by end user on the Users page.

Trace status comes from the spans, not from your agent. A trace is failed when at least one span has an error status. If the root span finished cleanly while a child step errored, the trace is badged recovered, and its errors are still counted. This matters when only some users are affected, because the framework can retry or answer around a failed step and the user still gets a bad result.

On a user’s page, two tiles read differently:

  • Failed traces counts traces whose root span ended in error.
  • Error rate counts traces with at least one errored span, recovered runs included.

A user with a high error rate and few failed traces is hitting errors the agent works around.

Set it up

  1. Make sure each run is attributed to a user. Open the Users page and read the line above the list, for example “400 of 471 traces carry an end user”. A large anonymous row means some code paths run outside a user scope. Wrap each call to the entrypoint in rius.user with an opaque id, not an email, so the root span carries it too.
  2. Wrap the entrypoint as an AGENT span and each tool body as a TOOL span. Install and init are covered in OpenAI Agents SDK, and all integrations are listed in Integrations.
import rius from agents import Agent, Runner, function_tool, set_tracing_disabled from rius import SpanKind rius.init(api_key="ri_...", service_name="my-agent") set_tracing_disabled(True) def lookup(email: str) -> str: return "active" # replace with your account lookup @function_tool @rius.observe(kind=SpanKind.TOOL) def account_status(email: str) -> str: """Fetch a customer's account standing by email.""" return lookup(email) support_agent = Agent( name="support_agent", instructions="Answer support questions; check accounts with the tool.", tools=[account_status], model="gpt-5-mini", ) @rius.observe(name="handle-ticket", kind=SpanKind.AGENT) def handle_ticket(ticket: str) -> str: result = Runner.run_sync(support_agent, ticket) return str(result.final_output) # Enter the user scope before the root span starts, so every span carries user.id with rius.user("user_8412"), rius.session("ticket-1093"): print(handle_ticket("Why is my account on hold?"))
  1. Keep @function_tool outermost and observe on the function body. The TOOL kind is what makes the Tool filter in the Spans tab work. A bare observe makes a CHAIN span, which the Tool Loop alert never counts.
  2. Scope each run with rius.session(), reusing your conversation or ticket id, so every turn of one conversation groups together.
  3. Find the pattern. On the Users page, sort by Failed. If a handful of users account for most failures, open one and compare their Error rate and cost per trace with the agent’s own numbers. Open a trace from Failed traces, or filter the trace list by that user, and read the failing span in the waterfall. The path from the root to the error is highlighted, and an error span leads with the exception and traceback.
  4. Compare the affected users. Look at their account settings, data size and language, and check whether their traces share a model. The Spans tab filters by kind, so choose Tool to see every tool call, or by status to see only failed spans.

What you get

  • A short list of affected users instead of an agent-wide failure rate, with the anonymous row showing how much traffic you cannot attribute.
  • For each user, the failing step: a model call named Response, or a named TOOL span for a tool such as account_status, with the exception and traceback.
  • Retry chains collapsed into one row, such as account_status · 3 attempts · recovered, when every attempt but the last failed.
  • A context growth strip on traces with several model calls, which plots input tokens per call. A user with a long history shows a runaway prompt here first.
  • Since previous llm call on the Input / Output tab, which shows what each step added to the prompt, if the span carries content.
  • A support ticket traced to a failing span: type the user’s exact id on the Users page, click Find, and open the trace from the time they wrote in.

To catch it next time, check the Users page sorted by Failed after each release. You can also set a custom alert on error rate filtered by Service or Agent. Alert filters do not include the user, so the alert tells you the failure rate moved and the Users page tells you who. For more on watching agents in production, see Monitor AI agents in production.

When something else fits

  • Handoffs produce no span of their own and every Responses API call is named Response, so a failure tied to one agent in a multi-agent run is visible only as consecutive generations. Add an observe wrapper per agent entry to name them.
  • If the failures follow a version or a model rather than a user, compare agent versions on the Agents page instead.
  • To read a single failing run in depth, start from Exploring traces and use the waterfall and map views.

Still stuck? Get help

Last updated on