How do I see the prompts and responses in an agent built with LlamaIndex?

Rius records the prompts and responses of a LlamaIndex agent for you once you install the LlamaIndex extra and call rius.init(). You read them on the Input / Output tab of a span in the trace. The retriever span shows the documents the model was given, and the model-call span shows the messages and the reply.

This integration is Python only.

How it works

With the llama-index extra installed, the LlamaIndex integration turns each query into a nested trace of typed spans, with no wrapper code from you:

• Retrieval steps become retriever spans. The span’s output shows what the model had to work with, which is where RAG debugging usually starts.
• Embedding calls become embedding spans.
• Synthesis model calls become LLM spans carrying the model, token usage and messages. Rius computes cost from these.

If the openai extra is installed too, a chat or completion call, streamed or not, also appears as a second LLM span under the LlamaIndex span. The call is counted once in tokens and cost, and the LlamaIndex span still shows the call’s total. This needs glassflow-rius 1.1.1 or later. Calls from MultiModalLLM models, embedding models and retrievers are not nested this way, so their OpenAI span starts a trace of its own.

What the integration does not see is your own code around the query engine. A request handler, a tool function or a routing step is not a LlamaIndex span, so it appears only if you wrap it yourself with rius.observe. A bare observe makes a chain span named after the function. A tool function that no integration already records takes kind=SpanKind.TOOL, so it shows as a tool span.

Content capture is on by default. Setting RIUS_CAPTURE_CONTENT to false, or using the capture_content and mask options, strips prompt and response content before it leaves your process. Document ids and scores stay visible even then. For the full list of integrations and their blind spots, see Integrations.

Set it up

1. Install the instrumentation next to LlamaIndex. The extra installs the instrumentation, not llama-index itself, so name the package for a fresh environment:

pip install "glassflow-rius[llama-index]" llama-index

2. Call init() once at startup, then build your indexes and query engines exactly as before:

import rius
 
rius.init(api_key="ri_...", service_name="my-agent")
# build indexes and query engines exactly as before

3. Optional: give each request a root span of your own so the whole run, not only the query, is one trace. query_engine here is your own object:

@rius.observe
def handle(query: str) -> str:
    return str(query_engine.query(query))

4. Run one query, then open Traces in the console and click the trace. If the LlamaIndex extra is installed without llama-index in the environment, init() logs a DependencyConflict and the integration stays off, so install both.

5. Read the content:

   • In the waterfall, spans are badged by kind. Select the retriever span and open Input / Output to see what was retrieved.
   • Select the LLM span and open Input / Output to see the prompt and the completion. Raw shows the span exactly as ingested.
   • On an LLM span, Since previous llm call marks which messages were added, removed or changed since the trace’s previous model call.
   • On the Spans tab, the All kinds menu narrows the list to Retriever or LLM spans. Inside a trace, ]l and [l jump to the next and previous LLM span.

The exploring traces page covers the trace list, filters and the span panel in full. The wider workflow is in Monitor AI agents in production.

What you get

• The prompt and the completion for every model call in the query, on the span’s Input / Output tab.
• The retrieved documents, on the retriever span, so you can tell whether a bad answer came from retrieval or from synthesis.
• Model, token counts and a list-price cost estimate in US dollars on each LLM span. A call with no reported token usage shows a blank cost, not $0.
• A tree where retrieval, embedding and synthesis sit under the query, with failed spans highlighted from the root down to the error.
• A step-by-step view of how the prompt grows between model calls in an agent loop.
• Privacy you control: when content is stripped, the tab says so, and timings, tokens, cost, document ids and scores still show.

When something else fits

If your agent uses LlamaIndex.TS, there is no LlamaIndex integration in TypeScript. The OpenAI integration still traces the provider calls made through the OpenAI client, and manual generations cover the pipeline around them. If you do not want prompts and responses stored, turn content capture off and read structure, tokens and cost only.

Related