How do I see which step was slow in an agent built with LlamaIndex?

Rius shows which LlamaIndex step was slow by recording each query as a nested trace of retrieval, embedding and model-call spans, each with its own duration. You find the slow one in the trace’s waterfall or on the Map tab’s latency lens, or by sorting the Spans tab by Duration. Steps of your own, such as custom agent logic around the query engine, need an observe wrapper to get a span.

How it works

With the LlamaIndex integration installed, Rius turns a query into a nested trace of typed spans, with no code changes to your pipeline:

• Retrieval steps become RETRIEVER spans.
• Embedding calls become EMBEDDING spans.
• Synthesis model calls become LLM spans with the model, token usage and messages. Rius computes the cost.

Every span has a start and an end, so every step has a duration. The slow step is the span that takes the most time.

The integration records what LlamaIndex does. It does not know about code that lives outside LlamaIndex, such as a function that prepares input, calls an external API or decides what the agent does next. That time appears inside the parent span, or as a gap, until you give the step a span of your own. A bare observe on a function makes a CHAIN span named after the function. For a tool function that no integration already records, observe with the TOOL kind (kind=SpanKind.TOOL) makes a span named after the tool, which the All kinds menu on the Spans tab can filter to.

The integration is Python only. There is no LlamaIndex integration in TypeScript. See Integrations for the full support matrix.

If the openai extra is also installed, an LLM call is traced as a second LLM span under the LlamaIndex span. The call is still counted once in tokens and cost, and the LlamaIndex span still shows the call’s total (glassflow-rius 1.1.1 or later).

Set it up

1. Install the integration and LlamaIndex itself. The extra installs only the instrumentation, so llama-index must be in your environment.

   pip install "glassflow-rius[llama-index]" llama-index

2. Call rius.init once at startup, build your indexes and query engines as before, and wrap the function that runs your agent step with observe. Name the function, because the span takes its name. Here query_engine is built from a local data folder; use your own.

   import rius
    
   from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
    
   rius.init(api_key="ri_...", service_name="my-agent")
    
   documents = SimpleDirectoryReader("data").load_data()
   query_engine = VectorStoreIndex.from_documents(documents).as_query_engine()
    
   @rius.observe
   def answer(question: str) -> str:
       return str(query_engine.query(question))

   The answer span becomes the parent of the retrieval, embedding and model spans for that query, so the whole step reads as one subtree. For a custom tool function that nothing else records, add observe with the TOOL kind instead of a bare one.

3. Run one query, then open the trace on the Traces page. See Exploring traces for the full tour.

4. Find the slow step:

   • Open the trace. The waterfall lays its spans out as a tree on a timeline, so a long bar is a slow step. LLM spans show their token counts inline.
   • On a deep trace, open the Map tab and pick the latency lens to see where the time went at a glance. Clicking a node lands you on the tree filtered to that node’s spans.
   • To compare steps across many runs, open the Spans tab, filter with the All kinds menu (for example Retriever, Embedding or LLM) and click the Duration header to sort.
   • Select a span and use the next and previous buttons, or j and k, to step through the spans one at a time. ]l jumps to the next LLM span.

To set up Rius from scratch, follow the SDK installation page linked from Integrations. For the wider picture of watching agents in production, see Monitor AI agents in production.

What you get

• A nested trace for each LlamaIndex query, with a duration on every retrieval, embedding and model-call span.
• Model, token usage and computed cost on each LLM span. Costs are list-price estimates in US dollars.
• On the retriever span, the output shows what the model was given to work with, which is where RAG debugging usually starts. Retrieved content and messages are content attributes, so the SDK’s privacy controls cover them. Document ids and scores stay visible even with content stripped.
• A cross-run view of steps on the Spans tab, sortable by duration and filterable by kind, status and the same filters as the trace list.
• A named span for each step of your own that you wrap with observe, so slow code outside LlamaIndex is visible as its own bar.
• Compare runs on an open trace to see which steps changed and where latency moved.

When something else fits

• If your agent is built in TypeScript with LlamaIndex.TS, there is no LlamaIndex integration. The OpenAI integration still traces the provider calls, and manual generations cover the pipeline around them.
• If you need to know whether a process is stuck rather than slow, remember that a trace shows finished steps. Turn on partial spans so a run still in flight shows as running.

Related