How do I trace streaming responses from the Anthropic API?

Rius traces streamed Anthropic calls automatically with model, token usage and cost, and a manual generation adds time to first token. Install the Anthropic extra, call init(), and use your client as before. If time to first token matters for a call, wrap that call in a manual generation and call record_first_token() from your streaming loop.

How it works

• Automatic tracing. With the Anthropic integration on, each call becomes a span for the model call. It carries the model, the request and response messages, and token usage, including cache reads and writes. Cost is computed by Rius from the model and tokens, so you never send it.
• Streamed calls. Token usage is captured on streamed calls too, because Anthropic reports it in the message lifecycle events. A stream therefore gets tokens and a cost estimate like any other call.
• Time to first token. Auto-instrumented Anthropic streams do not record it, in Python or TypeScript. Python openai streams record it automatically; Anthropic streams and TypeScript OpenAI streams do not.
• Manual generation. A manual generation records time to first token when your loop calls record_first_token() (recordFirstToken() in TypeScript). That call records the moment the first chunk arrives and marks the call as streamed. Only the first call records anything, so you can call it on every chunk. Rius derives time to first token from it, which gives you per-model percentiles and inter-token latency in your analytics.
• When the span ends. A manual generation stays open until you call end(). Call it after your loop has consumed the whole stream, in a finally block, so a failed stream still closes the span. Use record_exception() (recordException() in TypeScript) to record a failure you catch.

The Anthropic integration page covers what the automatic span captures. The tracing page covers manual generations.

Set it up

1. Install the instrumentation for Anthropic. The extra installs the instrumentation, not anthropic itself. In TypeScript, install @arizeai/openinference-instrumentation-anthropic at version 0.2.7 or newer instead. The Integrations page covers setup for every library.

pip install "glassflow-rius[anthropic]" anthropic

2. Call rius.init(api_key="ri_...", service_name="my-agent") once at startup, then stream with your Anthropic client exactly as before. This alone gives you tokens and cost on each streamed call.

3. For time to first token, trace the call as a manual generation and turn the Anthropic integration off for the process, or the call is counted twice: pass instruments=[] to rius.init(), or list only the integrations you keep. With it off, Anthropic calls you do not wrap yourself are no longer traced. Set provider to anthropic for every Claude call. With that provider, set_usage() adds the cache read and write counts to the input token count for you. messages and stream below are your own variables: your request messages and your Anthropic stream, created with stream=True.

gen = rius.start_generation(model="claude-haiku-4-5-20251001", provider="anthropic")
gen.set_input(messages)
 
parts, inp, out, cr, cw = [], 0, 0, 0, 0
try:
    for event in stream:
        gen.record_first_token()   # idempotent; call it unconditionally per event
        if event.type == "message_start":
            usage = event.message.usage
            inp = usage.input_tokens
            cr = usage.cache_read_input_tokens or 0
            cw = usage.cache_creation_input_tokens or 0
        elif event.type == "content_block_delta" and event.delta.type == "text_delta":
            parts.append(event.delta.text)
        elif event.type == "message_delta":
            out = event.usage.output_tokens
    gen.set_output("".join(parts))
    gen.set_usage(input_tokens=inp, output_tokens=out,
                  cache_read_input_tokens=cr, cache_write_input_tokens=cw)
finally:
    gen.end()

4. In TypeScript the same steps use startGeneration, recordFirstToken(), setUsage() and end(). Turn the integration off with an instruments list that leaves out anthropic, as in init({ instruments: [] }).

5. Open the trace in the console. The span shows the model, token counts and a cost, and a manual generation also shows time to first token.

What you get

• Every streamed Anthropic call appears as a span with the model, messages, token counts and a cost estimate. Costs are list-price estimates in US dollars.
• Cache reads and writes are counted, so cost does not undercount calls that use prompt caching.
• For calls you trace manually, time to first token appears per call, with per-model percentiles and inter-token latency in your analytics.
• Wrapping the call in observe nests the span inside your agent’s run. See Tracing your code.

When something else fits

• If you only need tokens, cost and errors for streamed Anthropic calls, the automatic integration is enough and needs no call-site changes.
• If latency to the first visible word matters for a call, trace that call as a manual generation. Python streams through the OpenAI integration already record time to first token automatically.

Related