How do I see the prompts and responses for calls to the OpenAI API?

Rius records the prompt and response of every OpenAI chat completion on the span for that call, and you read them on the Input / Output tab of the span in the trace. Install the OpenAI integration and call init(), and your existing client calls are traced with no changes to your call sites. Wrap your own function in observe if you want each call nested inside your agent’s run instead of showing as a standalone trace.

How it works

With the OpenAI integration on, init() instruments the client library for the whole process, so every call from anywhere in your code is traced. See OpenAI for the full recipe and Integrations for the other libraries.

What the integration records for each model call

• Each chat completion becomes one span for the model call.
• The span carries the model, token usage, and the input and output messages.
• It also carries the request parameters, finish reasons, the completion id and the tool definitions.
• Rius computes cost from the model and tokens. You never send a cost value.
• Streamed calls in Python also record time to first token.
• A call OpenAI rejects, such as a 404 for an unknown model, a 429 or a timeout, is recorded as an error span. Its error.type names the error class, for example openai.NotFoundError.

What needs your own span

The integration records the model call, not the work around it. A call made on its own shows up as a standalone trace. To see it inside your agent’s run, wrap the function that makes the call in observe. The model call then appears as a child of that span, so you read the prompt in the context of the step that sent it.

Where it shows in the trace

Open the trace and select the span for the model call. The Model section shows provider, requested and response model, tokens and cost. The Input / Output tab renders the prompt and the completion. Raw shows the span exactly as ingested.

Content capture follows your privacy controls. If you set capture_content=False (captureContent: false in TypeScript), the Input / Output tab says content was not captured. With a mask it shows the masked values. Timings, tokens and cost still show.

Set it up

1. Install the integration and wrap the function that calls OpenAI. In Python, run pip install "glassflow-rius[openai]" openai. In TypeScript, run npm install @arizeai/openinference-instrumentation-openai and call init() before you create the client. The extra installs the instrumentation, not openai itself.

import rius
from openai import OpenAI
 
rius.init(api_key="ri_...", service_name="my-agent")
client = OpenAI()  # use the client exactly as before
 
@rius.observe
def handle(query: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": query}],
    )
    return response.choices[0].message.content

2. If you stream responses, ask OpenAI to send usage with the final chunk. Without it, streamed spans have no token counts and so no cost. The prompt and response are still recorded.

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    stream=True,
    stream_options={"include_usage": True},
)

3. Run one call, then open Traces in the console and click your trace. Select the span for the model call and open Input / Output.
4. To find calls without opening each trace, use the Spans tab and filter the All kinds menu to LLM. Pick a span and read its Input / Output tab. The previous and next buttons keep that tab open, so you can read each call in turn. See Exploring traces for filters and the keyboard shortcuts.

What you get

• The full prompt and completion for each OpenAI call, on the Input / Output tab of its span.
• The model, token counts and a list-price cost estimate in US dollars on the same span, in the Model section.
• Nesting under your own observe span, so a prompt sits next to the step that sent it.
• Since previous llm call on the Input / Output tab. It shows what each call added to, removed from or changed in the prompt, which helps in an agent loop that appends to one conversation.
• Failed calls marked as error spans, with the error class in error.type.
• Searchable runs: the Spans tab matches span names, span IDs, trace IDs and session IDs. Attribute filters match exact values only, and prompts and completions are left out of suggested values, so type those yourself.

For how this fits into watching a whole agent in production, see Monitor AI agents in production.

When something else fits

• If you do not want prompt text to leave your process, set capture_content=False, or use a mask to replace the sensitive parts. You keep timings, tokens and cost.
• If you need time to first token on streamed OpenAI calls in TypeScript, trace the call as a manual generation, because the TypeScript integration does not record it.

Related