How do I see every tool call for calls to LiteLLM?

Rius records each LiteLLM completion as a span, and you wrap your tool functions with rius.observe so every tool call shows as its own span in the same trace. The model call comes from the integration. The tool calls come from your own wrappers.

How it works

Two layers make up the trace, and each one comes from a different place.

• The model call. With the litellm extra installed, each routed completion becomes a span with the model, token usage and messages. Rius computes cost on the server from the model and tokens. One extra covers every provider behind the router.
• The tool call. LiteLLM returns the model’s reply, and your code runs the tool. The integration does not see that step, so you wrap the tool function with rius.observe. Each call then becomes a span of its own, next to the model call that led to it. See Integrations for how run structure arrives from a thin layer of observe wrappers.
• Your root function. Wrap the function that runs the whole turn too, so the model call and the tool calls sit together under one root span.

If the openai extra is also installed, LiteLLM’s call to OpenAI-compatible providers appears as a second model span under the LiteLLM span. The call is still counted once in tokens and cost. The model name is stored without LiteLLM’s routing prefix when the prefix names the provider, so openai/gpt-4o-mini lands as gpt-4o-mini.

Set it up

1. Install the instrumentation and initialize Rius. The extra installs the instrumentation, not litellm itself. The LiteLLM integration page covers setup in full.

   pip install "glassflow-rius[litellm]" litellm

2. Wrap your root function and each tool function with rius.observe. The names handle and get_weather below are your own.

   import rius
   import litellm
   from rius import SpanKind
    
   rius.init(api_key="ri_...", service_name="my-agent")
    
   @rius.observe(kind=SpanKind.TOOL)
   def get_weather(city: str) -> str:
       return "sunny"
    
   @rius.observe
   def handle(query: str) -> str:
       response = litellm.completion(
           model="gpt-4o-mini",
           messages=[{"role": "user", "content": query}],
       )
       return get_weather("Paris")
    
   handle("What is the weather in Paris?")

   Pass kind=SpanKind.TOOL on each tool function. The span is then a TOOL span named execute_tool get_weather, which the Tool filter on the Spans tab and the Tool Loop alert pick up.

3. Run it, then open the trace on the Traces page. The waterfall shows your root span with the model call and the tool call beneath it.

4. To list every tool call across runs, open the Spans tab and pick Tool in the All kinds menu. Add filters for a service, agent or time range, and sort by Duration to find slow tools. The exploring traces page covers the filters.

5. Check the model call too. Select its span to see the model, token counts and cost, and open Input / Output to read the messages it carried.

For the wider picture of what to watch once traces arrive, see Monitor AI agents in production.

What you get

• A span for each LiteLLM completion with model, tokens, messages and a list-price cost estimate in US dollars.
• A span for each tool call you wrapped, placed in the waterfall at the point it ran, with timing and any recorded error. A failed span is highlighted along the path from the root.
• Retry chains that collapse into one row, such as fetch_article · 3 attempts · recovered, when the same tool is called back to back. Expand the row to see each attempt.
• A Spans tab view of every tool call in a time range, separate from the LLM calls.
• A context growth strip above the tree for traces with several model calls, showing input tokens per call in order.
• A Since previous llm call view on the Input / Output tab, which shows what each step added to the prompt in an agent loop.

When something else fits

• The integration is Python only. From TypeScript, calls to a LiteLLM proxy through the OpenAI client are traced by the OpenAI integration, and the model recorded is the one you ask the proxy for.
• If you run LiteLLM as a proxy server, the proxy is a separate process. Instrument the proxy itself or send its OpenTelemetry traces through the interoperability path.
• If a framework such as CrewAI drives LiteLLM, its own integration page covers what that framework records.

Related