How do I trace tool calls in LlamaIndex?

Rius traces a LlamaIndex run as nested spans, and you get one span for each tool call by wrapping the tool with observe and SpanKind.TOOL. The LlamaIndex integration records retrieval, embedding and model-call spans, and your tool spans sit in the same trace. Each tool span carries the tool name, its input and output, and any error.

How it works

With the LlamaIndex integration on, a query becomes a nested trace of typed spans:

• Retrieval steps appear as RETRIEVER spans.
• Embedding calls appear as EMBEDDING spans.
• Synthesis model calls appear as LLM spans with model, token usage and messages. Rius computes cost from them.

Tool calls get their own span when you wrap the tool function yourself. The tracing layer gives you observe, which traces each call to a function as a span. With kind=SpanKind.TOOL, the span is named execute_tool followed by the tool name, for example execute_tool get_weather, and it sets gen_ai.operation.name to execute_tool.

Nesting follows the current span. A span opened with observe or start_as_current_span becomes the current span, and spans started inside it nest under it. Wrap the function that handles a request with observe, and the tool spans and the spans of the run inside it share that parent.

A tool span records:

• gen_ai.tool.name, the tool’s own name. Without tool_name, observe uses the function’s name.
• gen_ai.tool.call.id, if you pass tool_call_id. This is the id the model put on the tool call, and it joins the tool span back to the model call that asked for it.
• gen_ai.tool.type, if you pass tool_type, such as function, extension or datastore.
• gen_ai.agent.name, the agent executing the tool: the innermost enclosing agent span opened with observe or the scoped span helper, else the configured agent name.
• The arguments in input.value and the return value in output.value, each truncated at 32 KB.

An exception is recorded on the span with ERROR status and error.type set to the exception class, and it is re-raised, so a failing tool groups with your other errors.

Set it up

1. Install the LlamaIndex instrumentation and call init() once, as the LlamaIndex integration page describes. Build your indexes and query engines as before.

pip install "glassflow-rius[llama-index]" llama-index

2. Wrap each tool function with observe, pass the name the model sees as tool_name, and pass tool_type if you want it recorded:

import rius
from rius import SpanKind
 
rius.init(api_key="ri_...", service_name="my-agent")
 
@rius.observe(kind=SpanKind.TOOL, tool_name="get_weather", tool_type="function")
def get_weather(city: str) -> str:
    ...

3. Hand the wrapped function to LlamaIndex as your tool. Every call now produces a tool span.

4. To record the model’s tool call id, open the span yourself. The id is known only at call time, so it is passed when the span is created:

with rius.start_as_current_span(
    kind=SpanKind.TOOL, tool_name="get_weather", tool_call_id=call_id
) as obs:
    obs.set_input(city)
    result = get_weather_impl(city)
    obs.set_output(result)

5. Wrap the function that handles each request with observe so the whole run has one root span.

6. Run one query that uses a tool and open the trace in the console. You see the run, the retriever, embedding and model spans, and your tool span in the same tree.

If you pass a span name on a tool span without a tool_name, the name doubles as the tool name and the SDK warns. Pass tool_name when the two should differ.

The LlamaIndex integration is Python only. In TypeScript, observe works the same way with kind: SpanKind.TOOL and toolName. See Integrations for what each language covers.

What you get

• One trace per run, with retrieval, embedding, model and tool spans nested together.
• A tool span for each call, named by the tool, with input, output, duration and error type.
• A link from each tool span back to the model call that requested it, when you pass tool_call_id.
• Tokens and cost on the model-call spans, counted once even when the OpenAI integration is also installed.
• Content controls: retrieved document content, messages and tool inputs and outputs are content, and the privacy controls cover them.

When something else fits

If you only need retrieval, embedding and model-call visibility, the LlamaIndex integration gives it with no wrapping. If your tools are called over MCP, the MCP integration traces those calls for you, so you do not wrap them by hand.

Related