Skip to Content

TanStack AI

Trace every TanStack AI  chat() run in Rius: the agent loop, each model call with its prompt, answer, tokens and cost, and each tool call with its arguments and result, grouped by conversation and by user.

What Rius does with those traces: it detects failing and silently degrading runs, such as tool loops, rising cost or latency, and quality drops, investigates the root cause, and hands it to your coding agent through MCP.

TanStack AI writes the spans itself, through its own otelMiddleware. Rius gives that middleware a tracer and maps what it writes onto the OpenTelemetry GenAI conventions, so there is no wrapper to add around your calls.

This integration is TypeScript only, because TanStack AI is. It needs @glassflow-ai/rius 1.9.0 or newer and was tested with @tanstack/ai 1.4.0.

Install

npm install @glassflow-ai/rius @opentelemetry/api

The middleware imports @opentelemetry/api, which @tanstack/ai does not install for you.

Set up tracing

Initialize Rius once, when your server starts, then create one middleware for the whole app. The API key comes from your workspace settings, as in the quickstart:

// tracing.ts import { init } from '@glassflow-ai/rius' import { trace } from '@opentelemetry/api' import { otelMiddleware } from '@tanstack/ai/middlewares/otel' export const rius = init({ apiKey: 'ri_...', serviceName: 'packing-assistant' }) export const otel = otelMiddleware({ // init() registered the tracer provider this tracer comes from tracer: trace.getTracer('packing-assistant'), // Record prompts, answers, tool arguments and tool results captureContent: true, })

Without captureContent: true the middleware records only metadata: models, token counts, tool names and timings.

Pass the middleware to each chat() call:

import { chat, maxIterations, toolDefinition } from '@tanstack/ai' import { openaiText } from '@tanstack/ai-openai' import { z } from 'zod' import { otel, rius } from './tracing' const getWeather = toolDefinition({ name: 'get_weather', description: 'Get the weather forecast for a city', inputSchema: z.object({ city: z.string() }), }).server(async ({ city }) => ({ city, temperature_c: 11, conditions: 'light rain' })) const stream = chat({ adapter: openaiText('gpt-5-mini'), messages: [{ role: 'user', content: 'What should I pack for Berlin tomorrow?' }], tools: [getWeather], agentLoopStrategy: maxIterations(6), threadId: 'trip-1842', middleware: [otel], }) for await (const chunk of stream) { if (chunk.type === 'TEXT_MESSAGE_CONTENT') process.stdout.write(chunk.delta) } // A script has to send what is still queued before it exits await rius.shutdown()

That is the whole setup. It works the same with any TanStack AI adapter, such as @tanstack/ai-anthropic.

What you see in Rius

Each chat() call is one run. It is a trace of its own, or part of the trace of a span that is already active around the call:

chat gpt-5-mini AGENT 9.4s ├─ chat gpt-5-mini #0 LLM 2.2s asks for the forecast │ └─ execute_tool get_weather TOOL 4ms └─ chat gpt-5-mini #1 LLM 7.2s answers
  • The run is an agent span. It carries the messages you sent, the final answer, the number of model calls (tanstack.ai.iterations) and the run’s token total.
  • Each model call is an LLM span with the provider, the requested and resolved model, the messages, the finish reason, and token usage including cached and reasoning tokens when the provider reports them. Cost is computed server-side from the model and the tokens.
  • Each tool call is a tool span with the tool’s name, its arguments and its result, under the model call that asked for it. A tool that throws is an error span with the exception.

Each model call is counted once in tokens and cost. TanStack AI reports a run’s token total on the run span as well as each call’s tokens on the call’s span; Rius keeps the run’s total for display and leaves it out of the sums.

Group runs by conversation and user

Set the session and the user through the middleware’s attributeEnricher. It runs for every span of a run, so every span carries both. Add it to the middleware in tracing.ts:

interface RequestContext { userId: string } export const otel = otelMiddleware({ tracer: trace.getTracer('packing-assistant'), captureContent: true, attributeEnricher: ({ ctx }) => { const attributes: Record<string, string> = {} // The threadId you pass to chat() is the conversation if (ctx.threadId) attributes['session.id'] = ctx.threadId // Image and speech calls run the enricher too, and have no context const userId = 'context' in ctx ? (ctx.context as RequestContext | undefined)?.userId : undefined if (userId) attributes['user.id'] = userId return attributes }, })

threadId is already required by chat(). The user comes from the context you pass to the call, which TanStack AI hands to every middleware hook:

// The handler of a chat route: a Request in, a streamed Response out import { chat, toServerSentEventsResponse } from '@tanstack/ai' import { openaiText } from '@tanstack/ai-openai' import { otel } from './tracing' export async function POST(request: Request): Promise<Response> { const { messages, threadId } = await request.json() const stream = chat({ adapter: openaiText('gpt-5-mini'), messages, threadId, // userIdOf stands for however your app authenticates a request context: { userId: await userIdOf(request) }, middleware: [otel], }) return toServerSentEventsResponse(stream) }

Every turn of a conversation then lands in one session, and cost, errors and latency are broken down per user.

Do not rely on withSession or withUser around a streamed chat() call. A streamed run starts its spans when the stream is first read. If that happens after the wrapper has returned, the spans carry neither. The enricher works wherever the stream is read.

Privacy

The middleware’s own captureContent decides what is recorded, and its redact option rewrites each text before it is recorded. On top of that, your Rius privacy controls apply to everything the middleware records: captureContent: false on init() strips prompts, answers, tool arguments and tool results before they leave your process, and a mask sees each of them.

Scripts and serverless functions

A long-running server needs nothing more: spans are exported in the background. A script, or a function that is frozen after it responds, has to flush first:

await rius.flush() // in a serverless function, once the run has ended await rius.shutdown() // in a script, before it exits

A run’s spans end when the run does. With stream: false that is when chat() resolves, so flush before you return. A streamed run ends when its stream has been read to the end, which for a streamed response is after your handler has returned. Flush in the callback your platform runs once the response has finished, such as after() in Next.js. A flush before that sends nothing.

Verify

Send one message through your chat route, then open the trace in the console. The run is an agent span named chat <model>, with an LLM span for each model call under it, and a tool span under the model call that asked for the tool. The LLM spans show token counts and a cost. The run span shows the run’s total and adds nothing to the workspace’s sums.

If nothing arrives, check the process log: the SDK logs one line when an export fails, which is usually a wrong API key. See Troubleshooting for the other causes. If the trace arrives without prompts and answers, the middleware is missing captureContent: true.

Known gaps

Most of these gaps come from what TanStack AI’s otelMiddleware records, not from Rius. Where an integration records the data, Rius shows it, for example time to first token on streamed calls.

  • A model call that only asks for tools shows no output. The middleware records the text of an answer, not the tool calls the model requested. The tool spans under that model call show what was called.
  • No time to first token. The middleware does not record when the first chunk of a stream arrived.
  • A model call that fails in a run with stream: false is not traced. chat() rejects before the middleware ends the failed call’s span and the run’s span. If the first model call fails, nothing arrives. If a later one fails, the earlier calls and their tools arrive without a run above them. A streamed run that fails is traced in full, with an error status.
  • A model call’s duration includes its tools. The middleware ends a model call’s span when the next model call starts, or when the run ends.
  • Your own spans inside a tool are not nested under the tool span. The middleware does not make its spans active, so a span you start in a tool joins whatever span was active around the chat() call.
  • Image, speech and transcription calls are not mapped. The middleware traces them too, and their spans arrive as it writes them. This page covers chat().
  • Spans sent to Rius without the SDK are counted twice. The step that counts each model call once runs inside the Rius SDK. To avoid counting tokens and cost twice, send the spans through the SDK, as in Set up tracing.

Next steps

  • Alerts: get notified about a recurring error, a tool stuck in a loop, or a metric crossing a threshold you set.
  • MCP server: let your coding agent read these traces and investigate a failure.
  • Tracing your code: add spans of your own around the steps TanStack AI does not see.
Last updated on