How do I see the prompts and responses for calls to LiteLLM?

Rius records the prompt and response of each LiteLLM call on its own span, and you read them on the Input / Output tab of that span in the trace. Install the LiteLLM extra, call rius.init(), and every completion routed through LiteLLM is traced, whichever provider it goes to. You add your own span only when you want the call to sit inside a named run of your agent.

How it works

Once rius.init() runs, the LiteLLM integration is on automatically. It is Python only, as is LiteLLM itself.

• What it records. Each routed completion becomes an LLM-kind span with the model, the token usage and the messages. Rius computes cost on its side from the model and the tokens.
• What needs your own span. The integration covers the model call itself. Run structure, such as a named step that contains the call, comes from the framework’s own spans or from a thin layer of observe wrappers. Wrap the function that makes the call with rius.observe if you want the LiteLLM span grouped under a named root. See Integrations for how the model-call layer and the observe layer fit together.
• Model names. When LiteLLM’s routing prefix names the provider, Rius stores the model without it. openai/gpt-4o-mini lands as gpt-4o-mini with the provider openai. A model filter written for the prefixed form needs updating.
• Second span for OpenAI-compatible providers. LiteLLM reaches OpenAI and OpenAI-compatible providers through the openai client library. If the openai extra is also installed, you see a second LLM span under the LiteLLM span. The call is still counted once in tokens and cost, and the LiteLLM span shows the run’s total (since glassflow-rius 1.0.2; some streamed setups are still counted twice).
• Content capture. Prompts and responses leave your process unless you turn content off, for example with RIUS_CAPTURE_CONTENT set to false. In that case the span keeps its timings, tokens and cost, and the Input / Output tab tells you content was not captured.

Set it up

1. Install the instrumentation and LiteLLM. The extra installs the instrumentation only, so name litellm for a fresh environment. Without litellm installed, init() logs a DependencyConflict and the integration stays off.

   pip install "glassflow-rius[litellm]" litellm

2. Initialize Rius before you make the call, then call LiteLLM as usual. Optionally wrap your own function with rius.observe, as below, so the call has a named parent.

   import rius
   import litellm
    
   rius.init(api_key="ri_...", service_name="my-agent")
    
   @rius.observe
   def answer(question: str) -> str:
       response = litellm.completion(
           model="gpt-4o-mini",  # or any provider LiteLLM routes to
           messages=[{"role": "user", "content": question}],
       )
       return response.choices[0].message.content

3. Open Traces in the console and select the trace. In the waterfall, find the LLM span. To jump straight to model calls, use the Spans tab and pick LLM in the All kinds menu.

4. Select the span and open the Input / Output tab. It renders the prompt and the completion. The Raw tab shows the span exactly as ingested.

5. If you call a LiteLLM proxy server from TypeScript with the OpenAI client, the OpenAI integration traces those calls. The model recorded is the one you ask the proxy for, not the provider it routes to. If you run LiteLLM as a proxy server rather than the Python library, the proxy is a separate process. Instrument the proxy itself or send its OpenTelemetry traces to Rius.

The full setup is on the LiteLLM integration page.

What you get

• The prompt and the response for every LiteLLM call, on the span’s Input / Output tab.
• A Model section on the same span with provider, requested and response model, input and output tokens, and cost. A call that reported no token usage shows a blank cost rather than $0.
• A Since previous llm call switch on that tab. It shows what changed in the input since the trace’s previous LLM call, with each message marked added, removed or changed. In an agent loop this shows what each step added to the prompt.
• Previous and next buttons, or the ]l and [l keys, to step through LLM spans and read each call’s input and output in turn.
• Attribute filters and span search to find calls again. Suggested filter values leave out prompts and completions, so type those values if you filter on them.
• A frozen shared view never carries span content, so sharing a trace list does not expose prompts or responses.

Exploring traces covers the trace list, the waterfall and the span panel in full. For the wider practice of watching agents run, see Monitor AI agents in production.

When something else fits

• If you want prompts and responses kept out of Rius, turn content capture off. The spans still carry timings, tokens and cost.
• If your agent is built on a framework that has its own Rius integration, check the Integrations matrix, since that framework may record the model call itself.

Related