Skip to Content

How do I see retries and fallbacks in an agent built with CrewAI?

Rius shows every model call a CrewAI agent makes as its own LLM span, and collapses retried tool calls that you wrap with observe into one retry row in the trace. A run that failed along the way but finished clean is badged recovered. Retries and fallbacks that happen in your own code need a span of their own to be visible, because CrewAI crews and tasks produce no spans.

How it works

What the integration records on its own

  • Each model call CrewAI makes arrives as an LLM span with model, tokens, cost and time to first token. CrewAI drives the OpenAI client, so the openai integration captures the calls. If LiteLLM is installed, CrewAI uses it instead, and you use the litellm integration.
  • Each call is a separate span, so an extra model call after a failure shows up as an extra LLM span in the tree. A span’s Model section shows the requested and response model, so a call that went to a different model is readable there.
  • The OpenAI client underneath retries a failed request on its own (max_retries, 2 by default). Those attempts happen inside one call, so they show as one LLM span, not as extra spans.
  • Generation spans are all named ChatCompletion. You tell them apart by position, model and timing.

What needs a span of your own

  • Crews and tasks produce no spans. A task that CrewAI runs again is not marked in the trace, so wrap your per-task entrypoints with observe if you need that.
  • Tools are not spans until you wrap the function body with observe and the TOOL kind. A retry loop or a fallback branch in your own code needs the same treatment: wrap each attempt, so every try is its own span.

Where it shows

  • In the waterfall, failed spans are highlighted along the path from the root to the error.
  • Back-to-back calls of the same tool with the same input collapse into one row such as fetch_article · 3 attempts · recovered, when every attempt but the last failed. Expand the row to see each attempt. Repeated calls that all succeeded stay separate rows.
  • A trace with failed child spans whose root span finished without an error is badged recovered. The errors are still counted, only the verdict changes.
  • On the Spans tab, the All kinds menu filters to Tool or LLM, so you can list every attempt across runs. Details are in Exploring traces.

Set it up

  1. Install the integration and initialize the SDK. The full setup is on the CrewAI integration page, and the other frameworks are listed under Integrations.
pip install "glassflow-rius[openai]" crewai
import rius rius.init(api_key="ri_...", service_name="my-agent")
  1. Wrap the attempt, not the loop. The function that does the work gets observe with the TOOL kind, and the retry loop calls it. The @tool decorator stays outermost, as on the CrewAI page. The names here are yours.
from crewai.tools import tool import urllib.request from rius import SpanKind def download(url: str) -> str: with urllib.request.urlopen(url, timeout=10) as response: return response.read().decode() @rius.observe(kind=SpanKind.TOOL) def fetch_article(url: str) -> str: return download(url) @tool("article_lookup") def article_lookup(url: str) -> str: """Fetch an article, trying up to three times.""" for attempt in range(3): try: return fetch_article(url) except Exception: if attempt == 2: raise
  1. Wrap the entrypoint around crew.kickoff() with observe and the AGENT kind, as the CrewAI page shows, so every attempt parents under one run.

  2. If your code switches to a backup model, put that switch inside its own observe wrapper with a clear name through the name argument, for example observe(name="fallback-model"). The wrapper span then marks where the fallback happened, and the LLM span under it shows the model that answered.

  3. Run a ticket that makes the tool fail once, then open Traces. Look for the retry row in the tree, and for the recovered badge on the trace.

What you get

  • Each model call in the crew as an LLM span with model, tokens and cost. Costs are list-price estimates in US dollars.
  • Each tool attempt as its own span, with failed attempts highlighted, and a retry loop that reads as one collapsed event instead of a wall of identical spans.
  • A recovered badge on runs that hit errors along the way and still finished, so you can tell a flaky run from a failed one.
  • A model fallback you can see by the model name on each LLM span, plus your own wrapper span marking the decision.
  • Filters on the Spans tab to find every tool attempt or every LLM call across runs.

When something else fits

  • If you only need to know whether runs succeed, the trace status and the recovered badge already answer that without extra wrappers.
  • If you need to know which task a model call belongs to, wrap per-task entrypoints with observe, since the integration does not show tasks.
  • For setting up the whole stack across several frameworks, start from the Monitor AI agents in production guide.

Still stuck? Get help

Last updated on