My LlamaIndex agent is slow to respond. Why?

Rius shows where a slow LlamaIndex run spends its time: the trace has a span for each retrieval, embedding and model call, and each span has its own duration. Open a slow trace and the longest span in the waterfall is your first suspect. The same duration data feeds the usage page and custom alerts, so the next slow run reaches you without a manual search. For fleet-wide monitoring, see Monitor AI agents in production.

How it works

With the LlamaIndex integration on, each query becomes a nested trace of typed spans:

• Retrieval steps are RETRIEVER spans. The span’s output shows what the model was given.
• Embedding calls are EMBEDDING spans.
• Synthesis model calls are LLM spans with model, token usage and messages. Rius computes cost from these.

Every span has a start and an end, so the trace is also a timeline of where the run waited. Slowness in a RAG pipeline usually comes from one of three places: the retrieval or embedding step, a model call with a long prompt, or a model call that is slow to start streaming. The spans separate these.

Two details help you read the timeline. If the openai extra is installed too, a chat or completion call appears as a second LLM span under the LlamaIndex span. The call is still counted once in tokens and cost. Calls from MultiModalLLM models, embedding models and retrievers are not nested, so their OpenAI span starts a trace of its own. If you look for a model call inside a slow query and cannot find it, check for a separate trace.

The integration is Python only. There is no LlamaIndex integration in TypeScript.

Set it up

1. Turn on the integration. This is the shared setup, covered in full on the LlamaIndex integration page and in Integrations. Build your indexes and query engines exactly as before.

pip install "glassflow-rius[llama-index]" llama-index

import rius
 
rius.init(api_key="ri_...", service_name="my-agent")
# build indexes and query engines exactly as before

2. Open Traces in the console and find a slow run. Each row shows latency, token total and cost, so look for the rows with the longest latency. The 7d preset is the default range. Narrow it with 15m or 1h if the slowness is recent.

3. Open the trace and read the waterfall. Look for the widest bar. If it is a RETRIEVER or EMBEDDING span, the delay is in retrieval. If it is an LLM span, the delay is in the model call. Note that the trace header gives the total duration, so you can compare the widest span against the whole run.

4. Switch to the Map tab and pick the latency lens. The map shows one node per span and highlights where the time went. Click a node to land on the tree filtered to that node’s spans.

5. Check the Context growth strip above the tree when the trace has several LLM calls. It plots input tokens per call in order, so a prompt that grows with every step, a common reason for later calls to run slower, shows up as rising bars.

6. To compare many runs at once, open the Spans tab. Filter with the All kinds menu to Retriever, Embedding or LLM, then click the Duration header to sort. This answers whether retrieval or generation is slow across the whole window, not just in one trace.

7. If a slow trace has a sibling that was fast, use Compare runs on the trace page to put them side by side and see which steps changed and where latency moved.

What you get

• The slowest step of a run, named and timed, in the waterfall and the Map latency lens.
• A cross-run answer from the Spans tab, sorted by duration and filtered to retrieval, embedding or model calls.
• The prompt behind a slow model call. Open the span’s Input / Output tab to see the messages, and use Since previous llm call to see what each step added to the prompt. Content is captured by default. If privacy controls strip it, timings, tokens and cost still work.
• Trends on Home, covered in Usage and cost. It shows average duration and latency percentiles (p50, p95, p99) for finished traces, and time to first token (p50 and p95) for streamed model calls. Time to first token is measured only on streamed calls, and the panel says what share of calls it covers.
• A way to catch it next time. Create a custom alert on duration or time to first token. Rules measure only the spans they match, so filter by Service or Agent to measure whole runs. To watch one step, filter by span kind instead. Deliver the alert to a webhook, email or Slack channel.

When something else fits

• If a trace is missing the retrieval or model spans entirely, the integration may be off. An extra installed without llama-index makes init() log a DependencyConflict and leaves it off.
• If the slow part is a process that never finishes, enable partial spans so the run shows as running. Heartbeats show liveness, not progress, so a hung process that still sends them shows as running.
• Rius traces, prices and alerts on runs. It does not change how your pipeline executes, so fixing the slow step is a change in your own LlamaIndex code.

Related