How do I see token counts for each call in an agent built with LlamaIndex?

Rius records token counts on the span for each model call in a LlamaIndex agent, with no span or attribute of your own. Install the LlamaIndex extra, call init, and open the trace to see tokens per call and a trace total. Rius computes cost from those counts, so you never send it. This works in Python only. For the wider setup, see Monitor AI agents in production.

How it works

The llama-index instrumentation turns a LlamaIndex query into a nested trace of typed spans:

• Retrieval steps become retriever spans.
• Embedding calls become embedding spans.
• Synthesis model calls become LLM spans, with the model, token usage and messages.

Rius computes cost on the server from the model and the token counts, so you send no cost value.

If the openai extra is installed as well, a chat or completion call, streamed or not, is also traced as a second LLM span under the LlamaIndex span. From glassflow-rius 1.1.1 the call is counted once in token and cost totals, and the LlamaIndex span still shows the call’s total. Calls from MultiModalLLM models, embedding models and retrievers are not nested this way: their OpenAI span starts a trace of its own. On earlier versions, or outside the supported instrumentation range, LLM calls are counted twice.

The integration is Python only. In TypeScript there is no LlamaIndex integration, so query-engine and retriever structure is not captured for LlamaIndex.TS. The OpenAI integration still traces the provider calls made through the OpenAI client.

Set it up

1. Install the instrumentation and LlamaIndex. The extra installs the instrumentation, not llama-index itself. Without llama-index in the environment, init() logs a DependencyConflict and the integration stays off.

pip install "glassflow-rius[llama-index]" llama-index

2. Call init() once, then build your indexes and query engines exactly as before. With a default global init(), every installed integration turns on automatically.

import rius
 
rius.init(api_key="ri_...", service_name="my-agent")
# build indexes and query engines exactly as before

3. Run one query, open Traces and open the trace. The pipeline appears as nested spans, and the LLM span at the end of each branch carries model, tokens and cost.

4. Read the counts where you need them:

   • In the waterfall, LLM spans show their token counts inline, and the trace header shows total tokens and cost.
   • Select an LLM span and its Model section lists input and output tokens and cost.
   • On the Spans tab, pick LLM in the All kinds menu and sort by the Tokens header to list calls by size.

5. If the agent has steps outside LlamaIndex that you want in the trace, wrap them with rius.observe, as in the quickstart. The model calls inside LlamaIndex need no wrapper.

The LlamaIndex integration page has the full setup and the known blind spots. The integrations overview lists every supported framework and how each is enabled.

What you get

• Token counts on each LLM span, with the model and messages, for every synthesis call in a query.
• A total for the whole run in the trace header, plus cost as a list-price estimate in US dollars.
• A Context growth strip above the tree for traces with several LLM calls. It plots input tokens per call in order, so a prompt that keeps growing shows up first.
• Retriever spans whose output shows what the model was given, which is where RAG debugging usually starts.
• A blank cost, not $0, on a call that reported no token usage, so it does not look free.

Retrieved document content and messages are content attributes, so Rius’s privacy controls cover them. Document ids and scores stay visible even with content stripped. The trace exploration guide covers the waterfall, the Spans tab and the span detail view in full.

When something else fits

• If your agent is written in TypeScript with LlamaIndex.TS, there is no LlamaIndex integration, so use the OpenAI integration for provider calls and manual generations for the pipeline around them.
• If you call a model provider directly, or use another framework, check the integrations overview for the page that matches it.

Related