How do I see one whole conversation in an agent built with LlamaIndex?

Rius shows a whole LlamaIndex conversation by grouping the traces of its queries under one session.id, which you set yourself. The LlamaIndex integration records each query as a nested trace on its own. Paste the session ID into the Spans tab, or follow the Session link in a trace header, to list every span in the conversation.

How it works

What the integration records on its own. Each query you run through LlamaIndex becomes a nested trace of typed spans:

• Retrieval steps are RETRIEVER spans. The retriever span’s output shows what the model was given to work with.
• Embedding calls are EMBEDDING spans.
• Synthesis model calls are LLM spans with model, token usage and messages. Rius computes cost from them.

If the openai extra is installed too, the model call also appears as a second LLM span under the LlamaIndex span. The call is still counted once in tokens and cost. See LlamaIndex for the full list of what lands in the trace.

What needs something of your own. The integration records queries, not conversations. A trace with no session.id is its own session, so the turns of a conversation stay separate traces until you give them a shared session ID. Rius groups traces by session.id. In Python you set it with rius.session. For a process that serves a single conversation, you can set RIUS_SESSION_ID for the whole process instead. Set session.id on every span, root included.

Where it shows. The session ID turns up in three places in the console:

• The Session link in a trace’s header opens the Spans tab filtered to that session.
• The search box on Spans matches session IDs, so pasting one lists every span in that session.
• The trace header shows the session, and the link lists every span in it.

Inside each trace, the waterfall shows the query as a tree on a timeline, so you can follow retrieval, embedding and synthesis in order. See Exploring traces for the views.

Set it up

1. Install the instrumentation and start Rius. The extra installs the instrumentation, not llama-index itself. Build your indexes and query engines exactly as before.

   pip install "glassflow-rius[llama-index]" llama-index

   import rius
    
   rius.init(api_key="ri_...", service_name="my-agent")
   # build indexes and query engines exactly as before

   The Integrations page covers installing and enabling every integration.

2. Give the conversation a session ID. In Python, set it with rius.session. If one process handles one conversation, set the process-wide variable before you start the agent. The value is your own conversation identifier.

   export RIUS_SESSION_ID=my-conversation-id

3. Run the turns of the conversation. Each query produces its own trace, and the turns that carry the same session.id belong to the same session.

4. Open the Traces page in the console and open one of the traces. Select the Session link in the header to see every span of the conversation. You can also go to Spans and paste the session ID into the search box.

5. Read the conversation. Open an LLM span and use Since previous llm call on the Input / Output tab. It shows what changed in the input since the trace’s previous LLM call, so you see what each step added to the prompt. A dropped turn or a rewritten system prompt stands out.

What you get

• Every span of one conversation in one filtered list, reachable from a trace header or by pasting the session ID.
• Each query as a nested trace, with retrieval, embedding and synthesis as separate spans, and the model, tokens and cost on the LLM span.
• A view of what each LLM call received, including the retriever’s output, so you can tell whether a bad answer came from retrieval or from the model.
• A diff of each call’s input against the previous call, with added, removed and changed messages marked.
• Context growth for a trace with several LLM calls: a strip above the tree plots input tokens per call in order, which is where a runaway prompt or an unbounded history shows up first.

Costs are list-price estimates in US dollars. Prompts and retrieved documents are content attributes, so the Rius privacy controls apply to them. Document IDs and scores stay visible even when content is stripped. For the wider picture of watching agents in production, see Monitor AI agents in production.

When something else fits

• The LlamaIndex integration is Python only. In TypeScript there is no LlamaIndex integration, so use the OpenAI integration for the provider calls and manual generations for the pipeline around them.
• If you only need to understand one query, the trace’s waterfall is enough and no session ID is needed.

Related