Skip to Content

How do I trace a document processing pipeline built with the Claude Agent SDK?

Rius traces a Claude Agent SDK document pipeline with a root span per document, a span per stage, and one generation filled from the run result. The Claude Agent SDK drives the Claude Code runtime in a separate process, so Rius has no automatic hook for it and you add these spans yourself. You get one trace per document with tokens, cost and timing, grouped by the session and user you set.

How it works

The Claude Agent SDK integration is manual, and it is available in Python. No model client runs in your process, so it has two parts:

  • An observe wrapper makes each task the root AGENT span.
  • A manual generation wraps the query() loop. You fill it from the run’s ResultMessage with token usage, the real model id and the collected answer. Rius computes cost from that.

A document pipeline needs more structure than one turn, so you add your own spans around the Claude call. See Tracing your code for the full reference.

  • A run root per document. One AGENT span named after the pipeline run, so each document is one trace.
  • A step per stage. Wrap each stage, such as text extraction or validation, in observe. A bare observe makes a CHAIN span named after the function, so give each stage a clear function name or a name.
  • A retrieval span. If a stage looks up reference material, open a RETRIEVER span with data_source_id and top_k, and record the hits with set_retrieved_documents(). It stores identifiers and scores, not document text.
  • A tool span for your own tool functions. A function you trace yourself and no integration records takes observe with kind=SpanKind.TOOL. A CHAIN span is never seen by the Tool kind in the Spans tab’s All kinds menu or by the Tool Loop alert.
  • A session and a user. Scope them around each document so every span carries session.id and user.id.

Set it up

  1. Install the SDK and the Claude Agent SDK. The claude CLI must be installed and authenticated.
pip install glassflow-rius claude-agent-sdk
  1. Wrap each stage, the retrieval, and the Claude turn. The functions search and parse_file stand for your own code. Use a full model id, not an alias like haiku, because an alias produces a generation with no cost. Keep provider="anthropic" on the generation and pass the cache counts as they come, and Rius adds them to the input tokens for you.
import rius from claude_agent_sdk import ( AssistantMessage, ClaudeAgentOptions, ResultMessage, TextBlock, query, ) from rius import SpanKind rius.init(api_key="ri_...", service_name="document-pipeline") MODEL = "claude-haiku-4-5" @rius.observe(name="extract-text") def extract_text(path: str) -> str: return parse_file(path) # your own parsing code @rius.observe(name="match-templates") def match_templates(text: str) -> list: with rius.start_as_current_span( kind=SpanKind.RETRIEVER, data_source_id="contract-templates", top_k=5 ) as obs: obs.set_input(text[:200]) docs = search(text) # your own search obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs]) return docs @rius.observe(name="summarise") async def summarise(text: str, templates: list) -> str: task = f"Summarise this document:\n{text}\n\nClosest templates: {[d.id for d in templates]}" options = ClaudeAgentOptions(model=MODEL, max_turns=1) parts = [] with rius.start_as_current_generation( "claude_agent_turn", model=MODEL, provider="anthropic", input=task ) as gen: async for message in query(prompt=task, options=options): if isinstance(message, AssistantMessage): parts.extend( b.text for b in message.content if isinstance(b, TextBlock) ) elif isinstance(message, ResultMessage): usage = message.usage or {} gen.set_usage( input_tokens=usage.get("input_tokens"), output_tokens=usage.get("output_tokens"), cache_read_input_tokens=usage.get("cache_read_input_tokens"), cache_write_input_tokens=usage.get("cache_creation_input_tokens"), ) models = list((message.model_usage or {}).keys()) if models: gen.set_response_model(models[0]) answer = "".join(parts) gen.set_output(answer) return answer @rius.observe(name="process-document", kind=SpanKind.AGENT) async def process_document(path: str) -> str: text = extract_text(path) templates = match_templates(text) return await summarise(text, templates) async def handle(document_id: str, requester_id: str, path: str) -> str: with rius.session(document_id), rius.user(requester_id): return await process_document(path)
  1. Choose the session id. Pass your own document or job id to rius.session() so every trace that touches one document correlates. The Claude Agent SDK’s session_id only arrives in the messages after the root span has started, so it can name the session only when you resume a stored conversation. Pass an opaque id to rius.user() rather than an email. Scope both per request, never as a process-wide default in a server that handles many users.

  2. Run one document and open the trace in the console. Check that the generation has a full model id, tokens and cost.

What you get

Each document becomes one trace with a readable shape:

process-document AGENT ├─ extract-text CHAIN ├─ match-templates CHAIN │ └─ retrieval contract-templates RETRIEVER └─ summarise CHAIN └─ claude_agent_turn LLM claude-haiku-4-5, tokens + cost
  • Timing per stage, so you see whether extraction, retrieval or the model turn is slow.
  • Tokens and cost on the model turn. Cost is a list-price estimate in US dollars, computed by Rius from the model id and your token counts, so you never send a cost value.
  • Retrieved document ids and scores on the retrieval span.
  • Sessions and users, so the console groups traces by document and by who requested them.

The Claude Code runtime’s inner work, such as its tool calls, sub-turns and file operations, produces no spans. The generation records the turn’s totals only. If a stage matters to you, run it in your own code and wrap it as above.

When something else fits

If your pipeline also calls a model client directly, for example with the Anthropic library, the Integrations page covers the automatic integrations that record those calls with no manual generation. If you only need one traced turn per task and no stages, the setup on the Claude Agent SDK page is enough.

Still stuck? Get help

Last updated on