How do I trace a multi-agent workflow built with LlamaIndex?

Rius traces a LlamaIndex multi-agent workflow as one trace per run, with each agent wrapped in observe as its own span and the LlamaIndex calls nested inside it. You wrap the run and each agent, and the LlamaIndex integration fills in the retrieval, embedding and model calls beneath them. You then read the order of the agents in the trace waterfall. The integration is Python only.

How it works

Two layers combine into one trace:

• The LlamaIndex integration records what happens inside each agent. Retrieval steps become RETRIEVER spans, embedding calls become EMBEDDING spans, and model calls become LLM spans with model, token usage and messages. Rius computes cost from the model and token counts. See LlamaIndex for the full list.
• Your observe wrappers record the structure of the run. Spans nest through parent context, so a wrapper around the whole run becomes the root span. A wrapper around each agent becomes a child of that root, and the integration’s spans nest under the agent that made the calls.

A trace is one end-to-end run with exactly one root span, so the workflow’s duration and status are the root’s. The span kinds include AGENT for an agent invocation, as described in Concepts.

The wrapper is what makes an agent visible as a span. Name each wrapped function after its agent, such as planner or researcher, and that name is what you see in the waterfall. If the workflow calls the OpenAI client through an LLM, install the openai extra too. The call is traced as a second LLM span under the LlamaIndex span, and it is still counted once in tokens and cost.

Set it up

1. Install the instrumentation and LlamaIndex. The extra installs the instrumentation, not llama-index itself:

   pip install "glassflow-rius[llama-index]" llama-index

2. Call rius.init once at startup, then wrap each agent’s entry function and the function that runs the whole workflow. The names below are examples of your own functions, so replace them with your agents:

   import rius
   from rius import SpanKind
    
   rius.init(api_key="ri_...", service_name="my-agent")
    
   @rius.observe(kind=SpanKind.AGENT)
   def planner(task: str) -> str:
       # call your LlamaIndex planner agent here
       ...
    
   @rius.observe(kind=SpanKind.AGENT)
   def researcher(plan: str) -> str:
       # call your LlamaIndex research agent here
       ...
    
   @rius.observe
   def run_workflow(task: str) -> str:
       plan = planner(task)
       return researcher(plan)

3. Call the outermost function once per run, so that every agent in the run shares one trace. If the workflow spans several turns of a conversation, set a session id so the traces group together. Concepts explains sessions.

4. Run the workflow once and open the trace in the console. You should see the workflow as the root, each agent as a child span, and the retriever, embedding and model spans beneath the agent that triggered them.

With a global init(), every installed integration turns on automatically, so you do not list LlamaIndex anywhere. The Integrations page covers installing and enabling in more detail.

What you get

• One trace per run. The root span’s duration and status describe the whole workflow. A failed agent shows as a failed run, and the waterfall highlights the path from the root to the failing span.
• Each agent as its own span. You can see how long each agent took and which model calls, retrievals and embeddings happened inside it.
• Handoffs you can read. Because each agent is a span under the same root, the order and nesting of the spans show which agent ran after which, and which agent called which. The data you read is the wrapper spans you created, so the structure shows exactly the agent boundaries you chose.
• Cost and tokens per call. Cost is a list-price estimate in US dollars, computed by Rius from the model and token counts. A wrapper span shows the run’s total, and each call is counted once.
• Retrieval you can inspect. A retriever span’s output shows what the model was given to work with. Document ids and scores stay visible even if you strip content with the privacy controls.

When something else fits

• If you use LlamaIndex.TS, there is no LlamaIndex integration in TypeScript. The OpenAI integration still traces the provider calls, and manual generations cover the pipeline around them.
• If your agents run on a different framework, pick its page from Integrations. Several agent frameworks are listed there with their own setup.

Related