How do I trace LiteLLM in Python?

Rius traces LiteLLM in Python with one extra: install glassflow-rius[litellm], call rius.init() once, then call litellm.completion as usual. Each routed completion appears as a span with the model, token usage, messages and a cost. One extra covers every provider LiteLLM routes to, so switching providers leaves your tracing unchanged.

How it works

The litellm extra installs the instrumentation for LiteLLM. When you call rius.init() globally, Rius turns on every integration whose library is installed, including LiteLLM. You do not wrap your calls or add spans by hand.

• One span per routed call. Each completion becomes a span for the model call, carrying the model, token usage and messages.
• Cost is computed by Rius. You never send a cost value. Rius prices the call from the model and token counts, as a list-price estimate in US dollars.
• Model names lose the routing prefix. When the prefix names the provider, the model is stored without it (since glassflow-rius 1.0.1). openai/gpt-4o-mini lands as gpt-4o-mini with the provider openai. Update any model filters that matched the prefixed form.
• Calls are counted once. LiteLLM reaches OpenAI and OpenAI-compatible providers through the openai client library. If the openai extra is installed too, that client call is traced as a second span under the LiteLLM span. Since glassflow-rius 1.0.2, tokens and cost for that call are counted once, and the LiteLLM span still shows the run’s total.

The integration is Python only, as is the LiteLLM library itself. The full list of integrations and their language support is on Integrations.

Set it up

1. Install the SDK with the LiteLLM extra. The extra installs the instrumentation, not litellm itself, so the command names litellm for a fresh environment. A project that already uses LiteLLM keeps its installed version. The SDK needs Python 3.10 or newer.

   pip install "glassflow-rius[litellm]" litellm

2. Call rius.init() once, before your first LiteLLM call, then make a completion. Replace ri_... with your API key and my-agent with the name you want to find your agent under.

   import rius
   import litellm
    
   rius.init(api_key="ri_...", service_name="my-agent")
    
   response = litellm.completion(
       model="gpt-4o-mini",  # or any provider LiteLLM routes to
       messages=[{"role": "user", "content": "hi"}],
   )

3. Optionally limit what gets instrumented. By default init() enables every installed integration. Pass instruments=["litellm"] to restrict it to LiteLLM, or instruments=[] to turn all of them off.

4. Run the script and open the trace in the console. The span shows the model, the token counts and a cost.

If init() logs a DependencyConflict, the extra is installed but litellm is not in your environment, and the LiteLLM integration stays off. Install litellm and run again. The full list of init() arguments and environment variables is on Installation and configuration.

What you get

• A span for every routed completion, with the model, token usage and messages.
• A cost on each call, computed by Rius from the model and tokens. Costs are list-price estimates in US dollars from LiteLLM’s public model-price catalog, not billed amounts. A model the catalog does not know gets an empty cost, not zero.
• Model analytics across every provider you route to, so you can send the same call to a different provider and compare the two.
• Token and cost totals that count each call once, with the LiteLLM span still showing the run’s total.
• Stable tracing when you change providers, because LiteLLM presents them all behind one API.

A few setups, such as streamed calls on some instrumentation versions, can still count a call twice. The LiteLLM integration page covers those cases. Its full setup and verification steps are on LiteLLM.

When something else fits

• If you run LiteLLM as a proxy server rather than as the Python library, the proxy is a separate process. Instrument the proxy itself, or send its OpenTelemetry traces to Rius.
• From TypeScript, calls to a LiteLLM proxy through its OpenAI-compatible endpoint are traced by the OpenAI integration, because you make them with the OpenAI client. The model recorded is the one you ask the proxy for, not the provider it routes to.

Related