How do I trace a customer support agent built with LlamaIndex?

Rius traces a LlamaIndex customer support agent by installing the llama-index extra and calling init, which records retrievers, embeddings and model calls with tokens and cost. You then add a few observe wrappers for the run root, each stage and your tool functions, and scope a session and user around every message. This works in Python only.

How it works

Two layers produce the trace.

• The LlamaIndex integration. After rius.init(), a query becomes a nested trace of typed spans. Retrieval steps are retriever spans, embedding calls are embedding spans, and synthesis model calls are model spans with model, token usage and messages. Rius computes cost on its side, so you never send it. The retriever span’s output shows what the model was given to work with, which is usually where a wrong support answer starts.
• Your own spans. Your agent logic sits around the query engine: deciding what the customer wants, calling order or account tools, drafting the reply. LlamaIndex does not see that, so you wrap it with observe. A bare observe makes a span named after the function. The tool kind makes a span named after the tool, and the agent kind makes a span for the agent run.

If the openai extra is installed too, the model call appears as a second model span under the LlamaIndex span. The call is still counted once in tokens and cost, so do not add a manual generation for calls LlamaIndex already records.

The session ties the traces of one conversation together. A trace covers one message, and a session is the id you assign to the whole conversation. The user is the customer, so you can see failures and cost per customer.

Set it up

1. Install the instrumentation and call init. The extra installs the instrumentation, not LlamaIndex itself. The LlamaIndex integration page covers setup and what is captured in full.

pip install "glassflow-rius[llama-index]" llama-index

import rius
 
rius.init(api_key="ri_...", service_name="my-agent")
# build indexes and query engines exactly as before

2. Wrap the run root, one function per stage, and each tool. The function names below are your own. Keep each LlamaIndex query inside the stage that calls it, so its retriever and model spans nest under your stage.

import rius
from rius import SpanKind
 
@rius.observe(kind=SpanKind.TOOL)
def lookup_order(order_id: str) -> str:
    ...
 
@rius.observe
def retrieve_and_draft(question: str) -> str:
    ...  # your LlamaIndex query engine call
 
@rius.observe(kind=SpanKind.AGENT)
def support_agent(message: str) -> str:
    ...  # calls your stages and tools

The tool kind matters. A tool traced with a bare observe becomes a generic step span, which the Tool kind filter in the Spans tab and the Tool Loop alert never see.

3. If you search a help center or knowledge base outside LlamaIndex, open a retrieval span yourself. Skip this when the search runs through a LlamaIndex retriever, because that span already exists.

with rius.start_as_current_span(
    kind=SpanKind.RETRIEVER, data_source_id="product-kb", top_k=5
) as obs:
    obs.set_input(query)
    docs = search(query)
    obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs])

4. Set the user and session around each incoming message. Use your own conversation id, and prefer an opaque customer id over an email.

with rius.user(customer_id):
    with rius.session(conversation_id):
        reply = support_agent(message)

Do not set a process-wide session in a server that handles many customers, because every conversation would merge into one session. Scope each request instead. See Integrations for how the other frameworks attach.

What you get

• One trace per customer message. The agent span is the root, your stages sit below it, and the retriever, embedding and model spans from LlamaIndex nest inside the stage that ran them.
• Tokens and a list-price cost estimate in US dollars on every model call, counted once.
• Tool calls under their own tool names, so you can filter them by kind in the Spans tab and have the Tool Loop alert watch them.
• Document ids and scores on retrieval spans. They stay visible even when content capture is off, while retrieved text and messages are content and follow your privacy settings.
• Conversations grouped by session and customers listed on the Users page, with their traces, failures and cost.

When something else fits

• If your support agent runs on LlamaIndex.TS, there is no LlamaIndex integration in TypeScript. The OpenAI integration still traces its provider calls, and manual generations cover the rest.
• If you only need model-call tokens and cost with no run structure, the integration alone is enough and the wrappers are optional.

Related