How do I trace a data analysis agent built with CrewAI?

Rius traces a CrewAI data analysis agent by capturing the framework’s model calls through the openai integration and by adding observe wrappers for the run, its stages, its retrieval and its tools. Install glassflow-rius[openai], call rius.init(), then wrap your entrypoint, each analysis stage and each tool body. Set a session and a user around each run so you can find it later.

How it works

CrewAI 1.x drives the OpenAI client natively, so the openai integration captures its model calls. If LiteLLM is installed, CrewAI uses it instead, and you use the litellm integration. The CrewAI integration page covers both cases, and the Integrations page lists every supported framework.

The crew structure runs inside CrewAI, where auto-instrumentation cannot see it. Crews and tasks produce no spans of their own, so a model call does not show which task it belonged to. You add that shape with observe, and a data analysis agent usually needs four kinds of span:

• A run root. An AGENT span around the function that calls crew.kickoff(). Every other span of the run nests under it.
• A step per stage. A plain observe wrapper (a CHAIN span named after the function) for each stage, such as loading data, analyzing and summarizing. Wrap per-task entrypoints this way when you want to see which task a model call belongs to.
• Retrieval. A RETRIEVER span for each lookup of schema notes, metric definitions or documents. Pass data_source_id and top_k, and record the returned document ids and scores with set_retrieved_documents(). These are metadata, so they stay even when content capture is off.
• Tools. The tools that query a warehouse or run code get observe with kind=SpanKind.TOOL on the function body. A bare observe makes a CHAIN span, which Rius does not treat as a tool.

Model calls arrive as LLM generations with the model. Token counts, and so cost, appear when the call reports usage. A streamed call needs usage enabled on the underlying OpenAI client to carry token counts, see the include_usage note on the OpenAI integration page. Rius computes cost from the model and tokens, so you never send a cost value.

Set it up

1. Install the SDK with the OpenAI instrumentation, plus CrewAI. The full SDK setup is on the CrewAI integration page.

pip install "glassflow-rius[openai]" crewai

2. Initialize Rius once, wrap the tools, the stages, the retrieval and the run root. The names run_query, fetch_definitions, analyze, summarize, analyze-dataset and metrics-kb are your own. Put @tool outermost and observe on the function body.

import rius
from crewai import Agent, Crew, Task
from crewai.tools import tool
from rius import SpanKind
 
rius.init(api_key="ri_...", service_name="data-analysis-agent")
 
@tool("run_query")
@rius.observe(kind=SpanKind.TOOL)
def run_query(sql: str) -> str:
    """Run a read-only SQL query and return the rows."""
    return execute(sql)
 
analyst = Agent(
    role="Data analyst",
    goal="Answer questions about the dataset accurately.",
    backstory="You query data and explain the results.",
    tools=[run_query],
    llm="gpt-5-mini",
)
 
def fetch_definitions(question: str) -> list:
    with rius.start_as_current_span(
        kind=SpanKind.RETRIEVER, data_source_id="metrics-kb", top_k=5
    ) as obs:
        obs.set_input(question)
        docs = search(question)
        obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs])
    return docs
 
@rius.observe(name="analyze")
def analyze(question: str, docs: list) -> str:
    task = Task(
        description=(
            f"Answer this question using the dataset: {question}\n\n"
            "Metric definitions:\n" + "\n".join(d.text for d in docs)
        ),
        expected_output="The figures and a short explanation.",
        agent=analyst,
    )
    return str(Crew(agents=[analyst], tasks=[task]).kickoff())
 
@rius.observe(name="summarize")
def summarize(findings: str) -> str:
    return format_report(findings)
 
@rius.observe(name="analyze-dataset", kind=SpanKind.AGENT)
def run(question: str) -> str:
    docs = fetch_definitions(question)
    return summarize(analyze(question, docs))

3. Scope each run with a session and a user. Pass your own job or conversation id to rius.session() and an opaque user id to rius.user(), so every span of the run carries both. Without a session, each trace is its own session, and without a user it counts as anonymous.

with rius.session(job_id), rius.user(user_id):
    run("Which region had the largest drop in revenue last quarter?")

4. If the agent runs where no terminal is attached, such as CI or a container, set CREWAI_TRACING_ENABLED=false. CrewAI otherwise asks about its own tracing on first run.

5. Run one question that calls a tool, then open the trace in the console and check for the root AGENT span, the stage steps, the retrieval span, the LLM generations and your named TOOL spans.

What you get

• One trace per analysis run, with the analyze-dataset root and your stages nested under it.
• A retrieval metrics-kb span for each lookup, with the ids and scores of the documents returned.
• An LLM generation for each model call CrewAI makes, with the model and, when the call reports usage, token counts and a Rius-computed cost estimate. Generation names are generic, such as ChatCompletion.
• A TOOL span for each query or code execution, named after the tool, so you see which call failed or ran long.
• Runs grouped by session and traced to a user, so you can follow one job or one person across traces.

When something else fits

If you only need model calls with tokens and cost, rius.init() with the openai integration is enough and you can skip the wrappers. If your crew runs on LiteLLM, follow the litellm integration instead of openai. For a different framework, pick its page from Integrations.

Related