How do I trace a customer support agent built with CrewAI?

Rius traces a CrewAI support agent by capturing the framework’s model calls through the openai integration and adding observe wrappers for the run, its tools and each stage. Install glassflow-rius[openai], call rius.init(), wrap the ticket entrypoint as an AGENT span, and every model call arrives with tokens and cost. Add a retrieval span for your knowledge base lookup and scope each conversation with a session and the customer as the user.

How it works

CrewAI 1.x drives the OpenAI client natively, so the CrewAI integration captures each model call as a generation with model, tokens, cost and time to first token. If LiteLLM is installed, CrewAI uses it instead, and the litellm integration covers those calls.

The crew itself runs inside the framework, so crews and tasks produce no spans of their own. A support agent needs a few spans from your own code on top:

• A run root. Wrap the function around crew.kickoff() with observe and SpanKind.AGENT. It becomes the root of each ticket’s trace.
• A tool span per tool. Put observe with SpanKind.TOOL on the tool body, for example an account lookup. A bare observe makes a CHAIN span, which the Tool kind filter in the Spans tab does not show and the Tool Loop alert does not count.
• A retrieval span. Open a RETRIEVER span around your help center or knowledge base search. It records the data source, how many documents you asked for, and the ids and scores of what came back. It holds identifiers and scores, not document text, so it stays when content capture is off. Put the retrieved text in the span output if you want it captured as content.
• A step per stage. Wrap each stage you run outside the crew, such as gathering context or resolving the ticket, with a plain observe. Each becomes a CHAIN span named after the function. Because the trace does not show which task a model call belongs to, per-task entrypoints wrapped this way are the way to see it.

Spans nest through ambient context, so tool, retrieval and model spans parent under the run without extra wiring. See Tracing your code for every option.

Set it up

1. Install the SDK with the openai extra and CrewAI. Keep CrewAI’s own tracing prompt out of CI and containers by setting CREWAI_TRACING_ENABLED=false.

pip install "glassflow-rius[openai]" crewai

2. Initialize Rius once at startup, then build the tool, retrieval and stage wrappers. Put @tool outermost and observe on the function body. lookup, search, conversation_id and customer_id stand for your own code and values.

import rius
from crewai import Agent, Crew, Task
from crewai.tools import tool
from rius import SpanKind
 
rius.init(api_key="ri_...", service_name="support-agent")
 
@tool("account_status")
@rius.observe(kind=SpanKind.TOOL)
def account_status(email: str) -> str:
    """Fetch a customer's account standing by email."""
    return lookup(email)
 
support_agent = Agent(
    role="Support specialist",
    goal="Answer support tickets accurately and concisely.",
    backstory="You resolve product and account questions.",
    tools=[account_status],
    llm="gpt-5-mini",
)
 
@rius.observe(name="gather-context")
def gather_context(ticket: str) -> str:
    with rius.start_as_current_span(
        kind=SpanKind.RETRIEVER, data_source_id="help-center", top_k=5
    ) as obs:
        obs.set_input(ticket)
        docs = search(ticket)
        obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs])
    return "\n".join(d.text for d in docs)
 
@rius.observe(name="resolve-ticket")
def resolve_ticket(ticket: str, context: str) -> str:
    task = Task(
        description=f"Resolve this support ticket: {ticket}\n\nContext:\n{context}",
        expected_output="A short, direct answer.",
        agent=support_agent,
    )
    return str(Crew(agents=[support_agent], tasks=[task]).kickoff())
 
@rius.observe(name="handle-ticket", kind=SpanKind.AGENT)
def handle_ticket(ticket: str) -> str:
    return resolve_ticket(ticket, gather_context(ticket))

3. Scope each turn with the conversation as the session and the customer as the user. Both ids come from your application, and every span in the block carries them, including the model calls the integration records. Prefer an opaque customer id to an email.

with rius.session(conversation_id):
    with rius.user(customer_id):
        handle_ticket(message)

4. Run one ticket that calls a tool and open the trace in the console. You should see the AGENT root, the gather-context and resolve-ticket steps, the retrieval span, the model generations and your named tool span.

Spans export in batches. In a short script, Python’s exit hook sends what is queued when the process ends normally, and client.flush() before exit forces the export if you want to be sure.

What you get

• One trace per ticket, rooted at handle-ticket, with the stages, the retrieval, each model call and each tool call in order.
• Tokens and a list-price cost estimate in US dollars for every model call, computed by Rius from the model and token counts. You never send a cost value.
• Retrieval spans that show which knowledge base was searched and which document ids scored highest.
• Tool spans the Tool kind filter and the Tool Loop alert can see, so a support agent that keeps calling the same lookup is caught.
• Conversations grouped by session, and a Users view of what each customer’s tickets cost and where they failed.

Generation spans are named ChatCompletion, so rely on your own span names to tell stages apart. Streamed calls carry token counts only when usage is enabled on the underlying OpenAI client.

When something else fits

• If your support agent mixes frameworks or languages, the integrations overview shows which integration covers each part. TypeScript has no CrewAI integration, so trace those model calls as manual generations.
• If you only want the model calls and not the stage structure, the integration alone gives you generations with tokens and cost, and you can add the wrappers later.

Related