My Pydantic AI agent fails only for some users. Why?
Rius shows which end users hit failures and the failing span inside each of their Pydantic AI traces, so you can see what those users have in common. Sort the Users page by Failed, open a failed trace, and read the span that errored. Compare those traces with a user whose runs succeed. Then add an alert so the next failure surfaces without waiting for a support ticket.
How it works
Pydantic AI emits its own OpenTelemetry spans, and Rius records them as they come. Each agent run is a root span named invoke_agent plus your agent’s name. Each model request is a chat span with the model, tokens and cost. Each tool call is an execute_tool span named after the tool. See Pydantic AI for what lands where.
A trace is failed when at least one span carries an OpenTelemetry ERROR status. If the root span finished without an error of its own, the trace is badged recovered: a child step errored, the agent retried or fell back, and the run completed. Both kinds count as errors, so a user whose runs recover can still show a high error rate.
The Users page groups traces by user.id. A trace with no end user on any span counts as anonymous, and the line above the list shows how many of the window’s traces carry an end user. If most failing traces are anonymous, the cause is attribution, not the users. Some code paths are running outside a user scope.
What you can read inside a span depends on content capture. Pydantic AI’s InstrumentationSettings (for example include_content) controls what its spans carry, on top of the SDK’s privacy controls. If the Input / Output tab is empty, the span’s timings, tokens and cost are still there.
Set it up
- Put each request inside a user scope, next to the session scope, so Rius can group failures by user. Prefer an opaque user id to an email. The setup is covered in full on the Pydantic AI page.
import rius
from pydantic_ai import Agent
rius.init(api_key="ri_...", service_name="my-agent")
Agent.instrument_all(True)
support_agent = Agent(
"openai:gpt-5-mini",
instructions="Answer support questions; check accounts with the tool.",
)
with rius.user("user_8412"):
with rius.session("ticket-1093"):
result = support_agent.run_sync("Why is my account on hold?")
print(result.output)- Open Users and sort by Failed. If a handful of users account for most failures, the problem is specific to them. Check the line above the list first, and if the anonymous row is large, fix the missing user scope before drawing conclusions.
- Open one affected user. Compare their Error rate and cost per trace with the agent’s numbers on the Agents page. Then open a trace under Failed traces.
- In the trace, failed spans are highlighted along the path from the root to the error. Click the error span to read the exception and traceback. Note whether it is an
execute_toolspan or achatspan. A failing tool call points at that user’s data or account. A failing model request points at the prompt that user sent. - Check the Context growth strip on traces with several model calls. A user with long history shows rising input tokens per call. On a
chatspan, Since previous llm call shows what each step added to the prompt. - Compare a failed trace with a successful run of the same agent using Compare runs, to see which steps changed or were dropped. To test a pattern across users, open a span’s Raw tab and use Filter traces by this on an attribute such as the model. Then read the User column, which appears when the traces carry an end user, to see who is affected. The details are in Exploring traces.
What you get
- A list of the end users who fail most, and a user page with their failed traces, error rate, cost and sessions.
- The exact failing span for each user: the exception, the traceback, and, when content is captured, the prompt and tool arguments that triggered it.
- Retries shown as one row, such as a tool called three times with the last attempt recovering, so a flaky call reads as one event.
- A way to catch it next time. Rius has four pre-defined alerts, including Consistent Error Pattern and Swallowed Error, which counts errored child steps that are not tool calls. You can also create a custom alert on error rate. Filter it by service or agent to measure whole runs.
- Alert filters cover service, agent, step, model, span kind, operation, session and instance, not user. For per-user failures, use the Users page sorted by Failed, and set a filter on the trace list for a user you are watching.
For the broader routine, see Monitor AI agents in production.
When something else fits
- If the failing users are all in the anonymous row, add the user scope first. Until then the Users page cannot separate them.
- If your agent uses other libraries besides Pydantic AI, check the Integrations matrix so every model call arrives as a span.
- If the Input / Output tab is empty, adjust Pydantic AI’s
include_contentsetting or the SDK’s privacy controls. Without content you still see timings, tokens and the error type.