My CrewAI agent fails only for some users. Why?
Rius shows which end users your CrewAI runs fail for and which step fails, once you tag each request with the user. Sort the Users page by failed traces, open an affected user, and read the failing span in the trace. Compare what those users share, such as their data, account settings or language, with the users whose runs succeed. This page is part of Monitor AI agents in production.
How it works
The CrewAI integration records what happens inside each run:
- Your entrypoint wrapped with
observebecomes the rootAGENTspan. - Each model call becomes an
LLMgeneration with model, tokens and cost. The span is namedChatCompletion. - Each tool you wrap with
observeand the TOOL kind becomes aTOOLspan. - Crews and tasks produce no spans of their own. To see which task a failure belongs to, wrap per-task entrypoints with
observe.
Rius derives trace status from the spans. A trace is failed when any span carries an error status. If the root span finished cleanly after a child errored, the trace is badged recovered, which means CrewAI retried or fell back and the run still completed. Both kinds count as errors.
A span names its end user with user.id. The Users page groups finished traces by that user and shows each user’s traces, failed traces, error rate and cost. A user whose error rate or cost per trace is well above the agent’s average usually has a problem specific to them. Traces with no user fall into the anonymous row. See Users for the full page.
Set it up
-
Trace the crew with the tool and entrypoint wrappers. This is the setup from the CrewAI integration page, and Integrations lists the other frameworks.
pip install "glassflow-rius[openai]" crewaiimport rius from crewai import Agent, Crew, Task from crewai.tools import tool from rius import SpanKind rius.init(api_key="ri_...", service_name="my-agent") def lookup(email: str) -> str: return "active" # replace with your account lookup @tool("account_status") @rius.observe(kind=SpanKind.TOOL) def account_status(email: str) -> str: """Fetch a customer's account standing by email.""" return lookup(email) support_agent = Agent( role="Support specialist", goal="Answer support tickets accurately and concisely.", backstory="You resolve product and account questions.", tools=[account_status], llm="gpt-5-mini", ) @rius.observe(name="handle-ticket", kind=SpanKind.AGENT) def handle_ticket(ticket: str) -> str: task = Task( description=f"Resolve this support ticket: {ticket}", expected_output="A short, direct answer.", agent=support_agent, ) return str(Crew(agents=[support_agent], tasks=[task]).kickoff()) # Enter the user scope before the root span starts, so every span carries user.id with rius.user("user_8412"), rius.session("ticket-1093"): print(handle_ticket("Why is my account on hold?")) -
Scope each request with
rius.useraround the call tohandle_ticket, as the last lines show, passing your own user id. Prefer an opaque id to an email. Scope each kickoff withrius.session()too, so one conversation is one session. Without a user scope, those runs land in the anonymous row. -
Open Users and sort by Failed. If a handful of users account for most failures, look at what they have in common before changing anything that affects everyone. The line above the list shows how many traces carry a user, so you can tell how much of your traffic you can attribute.
-
Type an affected user’s exact ID, click Find, and open their page. Check Failed traces, then open a trace. In the waterfall, failed spans are highlighted along the path from the root to the error, and the detail panel leads with the exception and traceback. Press
]eto jump to the next failed span. -
Work out whether the failure is in a tool or in a model call. A failing
TOOLspan points at that tool’s input, often something in that user’s data. A failingLLMspan points at the call itself. The Context growth strip shows input tokens per call, so a long history for that user stands out. -
Compare a failed trace against a successful run of the same agent with Compare runs. The comparison shows which steps changed, were added or dropped, and where cost or latency moved.
-
Open a span’s Raw tab and use Filter traces by this on an attribute value, such as a model or a tool input. That tells you whether other failures share it.
What you get
- A ranked list of the users who fail most, with their error rate and cost compared against the agent’s average.
- A short path from a support ticket to the failing span: user, failed traces, trace, span.
- The exception and traceback on the failing step, and retry chains collapsed into one row such as
account_status · 3 attempts · recovered. - A way to catch it next time. Check Users sorted by Failed after a release. Rius also has four pre-defined alerts. Swallowed Error fires on errored child steps that are not tool calls. Tool Loop counts only spans of kind TOOL, which is why the tool in step 1 uses the TOOL kind. A tool wrapped with a bare
observeis never counted. For a custom error-rate alert, filter by agent to measure whole runs.
When something else fits
- If every user fails, the problem is not user-specific. Filter the Spans tab by status and read the common failing step, or compare the first failing trace against the last good one.
- If you need to know which CrewAI task failed, the framework records none, so add
observewrappers around each task entrypoint.