Skip to Content

How do I trace a RAG chatbot built with the Claude Agent SDK?

Rius traces a RAG chatbot built with the Claude Agent SDK through an observe root, a retrieval span you open and a generation filled from the run result. The Claude Agent SDK drives the Claude Code runtime in a separate process, so Rius coverage is manual and Python only. You get one trace per question, with the documents retrieved, the model, tokens and cost. Scope each conversation as a session and each request as a user.

How it works

The Claude Agent SDK runs no model client in your process, so auto-instrumentation has nothing to hook. Coverage comes from three pieces you add, described on the Claude Agent SDK integration page and in Tracing your code:

  • A run root. An observe wrapper with kind=SpanKind.AGENT makes each question the root span of its trace.
  • A retrieval span. A RAG chatbot has a step the framework never sees: the search over your knowledge base. Open a RETRIEVER span around it with data_source_id and top_k, and record the documents with set_retrieved_documents(). That call stores identifiers and scores, not document text, so it is metadata and survives content stripping. Put retrieved text in set_output() if you want it captured as content.
  • A generation around the query() loop. You fill it from the run’s ResultMessage: token usage, the real model id and the collected answer. Rius computes cost from those, so you never send a cost value.

Other stages, such as query rewriting, are an observe wrapper each and show as steps under the root. A step that reorders retrieved documents uses SpanKind.RERANKER. A tool function you trace yourself takes SpanKind.TOOL, because a bare observe makes a CHAIN span, which the Tool kind filter and the Tool Loop alert never see.

The Claude Code runtime’s inner work, such as its own tool calls and sub-turns, produces no spans. The generation records the turn’s totals only.

Set it up

  1. Install the SDK and the Claude Agent SDK. The claude CLI must be installed and authenticated. Full setup is on the Integrations page.

    pip install glassflow-rius claude-agent-sdk
  2. Wrap each question in an AGENT root, open a RETRIEVER span around your search, then run the model inside a generation. search is your own retrieval function, and the document attributes id and score are whatever your search returns.

    import rius from claude_agent_sdk import ( AssistantMessage, ClaudeAgentOptions, ResultMessage, TextBlock, query, ) from rius import SpanKind rius.init(api_key="ri_...", service_name="docs-chatbot") MODEL = "claude-haiku-4-5" @rius.observe(name="answer-question", kind=SpanKind.AGENT) async def answer_question(question: str) -> str: with rius.start_as_current_span( kind=SpanKind.RETRIEVER, data_source_id="product-kb", top_k=5 ) as obs: obs.set_input(question) docs = search(question) obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs]) prompt = question + "\n\nContext:\n" + "\n".join(d.text for d in docs) options = ClaudeAgentOptions(model=MODEL, max_turns=1) parts = [] with rius.start_as_current_generation( "claude_agent_turn", model=MODEL, provider="anthropic", input=prompt ) as gen: async for message in query(prompt=prompt, options=options): if isinstance(message, AssistantMessage): parts.extend( b.text for b in message.content if isinstance(b, TextBlock) ) elif isinstance(message, ResultMessage): usage = message.usage or {} gen.set_usage( input_tokens=usage.get("input_tokens"), output_tokens=usage.get("output_tokens"), cache_read_input_tokens=usage.get("cache_read_input_tokens"), cache_write_input_tokens=usage.get("cache_creation_input_tokens"), ) models = list((message.model_usage or {}).keys()) if models: gen.set_response_model(models[0]) answer = "".join(parts) gen.set_output(answer) return answer
  3. Use a full model id such as claude-haiku-4-5, not an alias such as haiku, for both the CLI option and the generation. The pricing table matches real model ids only, so an alias gives a generation with no cost. Keep provider="anthropic" and pass the cache counts as-is: set_usage() then adds them to input_tokens for you, which matters because agent loops are cache-heavy.

  4. Set the session and the user outside the wrapped function. observe takes no user argument, because a user belongs to each request, not to a function decorated once at import time.

    async def handle_turn(conversation_id: str, user_id: str, message: str) -> str: with rius.session(conversation_id), rius.user(user_id): return await answer_question(message)

    Use your own conversation id as the session id. The Claude Agent SDK’s session_id only arrives in the messages after the root span has started, so it can serve only when you resume a stored conversation. Prefer an opaque user id to an email. Do not set a process-wide session in a chatbot server that handles many users, because every user’s traces would merge into one session.

What you get

  • A trace per question, shaped like this:

    answer-question AGENT ├─ retrieval product-kb RETRIEVER └─ claude_agent_turn LLM claude-haiku-4-5, tokens + cost
  • The knowledge base searched, how many documents were requested and which document ids and scores came back, so you can tell a bad answer from a bad retrieval.

  • Tokens and a list-price cost estimate in US dollars on the generation, computed by Rius from the model and the token counts.

  • Conversations grouped as sessions, and each end user’s traces under that user.

  • To verify, run one question and open the trace in the console. The root, the retrieval span and the generation with a full model id, tokens and cost should be there. A generation without cost means an alias was used as the model id.

When something else fits

  • If you need to see the Claude Code runtime’s own tool calls and sub-turns, note that they produce no spans here. Trace the tools your chatbot owns with SpanKind.TOOL.
  • If your chatbot calls a model client such as OpenAI or Anthropic directly in your own process, that call is covered by auto-instrumentation, not by this manual setup.

Still stuck? Get help

Last updated on