What does Rius record when my agent calls LlamaIndex?

Rius records each LlamaIndex query as a nested trace of typed spans: retrieval steps, embedding calls and the model calls that write the answer. Each model call carries the model, token usage and messages, and Rius computes the cost from them. The integration is Python only, and you turn it on with one extra and a normal rius.init() call.

How it works

The llama-index instrumentation turns what happens inside a query into spans, with no wrapper code in your pipeline.

• Retrieval steps become RETRIEVER spans. The output of a retriever span shows what the model was given to work with, so it is the first place to look when an answer is wrong.
• Embedding calls become EMBEDDING spans.
• Synthesis model calls become LLM spans with the model, token usage and messages. Rius computes cost on its side from the model and token counts, so you never send a cost value. Costs are list-price estimates in US dollars, not billed amounts.

The spans nest the way the pipeline runs, so a query shows as one tree: the query on top, retrieval and synthesis below it, and the model call at the leaf.

If the openai extra is installed as well, an LLM’s chat or completion call, streamed or not, also appears as a second LLM span under the LlamaIndex span. Rius counts that call once in token and cost totals, and the LlamaIndex span still shows the call’s total. This needs glassflow-rius 1.1.1 or later. On earlier versions, or on instrumentation versions outside the supported range, the call is counted twice.

Set it up

1. Install the instrumentation next to LlamaIndex. The extra installs the instrumentation, not llama-index itself, so the package is named here for a fresh environment. A project that already uses LlamaIndex keeps its own copy.

pip install "glassflow-rius[llama-index]" llama-index

2. Call init() once at startup, then build your indexes and query engines exactly as before. With a default global init(), every installed integration turns on automatically, so there is nothing else to enable. Setup for all integrations is covered on the Integrations page.

import rius
 
rius.init(api_key="ri_...", service_name="my-agent")
# build indexes and query engines exactly as before

3. Run one query against an index and open the trace in the console. You see the pipeline as nested spans, with the LLM leaf carrying the model, tokens and cost.

4. If init() logs a DependencyConflict, the extra is installed without llama-index in the same environment, and the integration stays off. Install llama-index there and restart.

The LlamaIndex integration page has the full setup and what lands in the trace.

What you get

• A tree for each query. Retrieval, embedding and synthesis appear as separate typed spans, so you can see which step was slow or failed.
• Model, tokens and messages on each model call. Cost is computed from them, and cache reads and writes are priced at their own rates when they are reported.
• Retrieved context you can inspect. The retriever span output shows the documents the model received.
• Content controls. Retrieved document content and messages are content attributes. Setting RIUS_CAPTURE_CONTENT to false strips prompt and response content at export. Document ids and scores stay visible even with content stripped.

Known limits

• Some calls are not nested. Calls from MultiModalLLM models, embedding models and retrievers do not sit under the LlamaIndex span: their OpenAI span starts a trace of its own.
• There is no LlamaIndex integration in TypeScript, so query-engine and retriever structure is not captured for LlamaIndex.TS. The OpenAI integration still traces the provider calls made through the OpenAI client, and manual generations cover the pipeline around them.

When something else fits

If your pipeline is TypeScript, use the OpenAI integration for provider calls and manual generations for the rest. If you want a single nested trace for calls made outside LlamaIndex models, such as MultiModalLLM, expect those calls to appear as their own traces.

Related