How do I see token counts for each call to the OpenAI API?

Rius records input and output token counts on the span for each OpenAI call, with no changes to your call sites. Install the OpenAI integration, call init(), and open the trace in the console to see the tokens on each call. Streamed calls need include_usage set, and wrapping your call in observe shows it nested inside your agent’s run.

How it works

With the OpenAI integration installed, init() instruments the OpenAI client process-wide, so every call from anywhere in your process is traced. The OpenAI integration turns each chat completion into a span for the model call. That span carries:

• the model, and the resolved model name that OpenAI returns, for example gpt-4o-mini-2024-07-18
• input and output token counts
• the input and output messages, unless your privacy controls strip content
• the finish reasons, request parameters and tool definitions

Cost is computed by Rius from the model and the tokens. You never send it. It is a list-price estimate in US dollars, not a billed amount.

A call OpenAI rejects, such as a 404 for an unknown model, a 429 or a timeout, is recorded as an error span with error.type naming the error class, for example openai.NotFoundError.

You do not need your own span to get token counts. The integration records them on its own. Your own span matters for structure. A bare OpenAI call with nothing around it shows up as a standalone trace. Wrap the function that makes the call in observe and the call nests inside your agent’s run, so you read its tokens in context. This is the same pattern as in the integrations overview, where model calls arrive as generations with tokens and cost and run structure comes from observe wrappers.

Streamed calls are the one case that needs a setting. OpenAI does not include usage in a stream by default. Without include_usage, a streamed span has no token counts and so no cost.

Set it up

1. Install the integration and call init(). The extra installs the instrumentation, not openai itself. In TypeScript, install @arizeai/openinference-instrumentation-openai instead and call init() before you create the client. The integrations overview covers installation and enabling in full.

pip install "glassflow-rius[openai]" openai

2. Initialize Rius once, before you make calls, and use the OpenAI client as you did before. Wrap the function that makes the call in observe so the call nests inside your run:

import rius
from openai import OpenAI
 
rius.init(api_key="ri_...", service_name="my-agent")
client = OpenAI()  # use the client exactly as before
 
@rius.observe
def handle(messages):
    return client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
    )

3. For streamed calls, ask OpenAI to send usage with the final chunk:

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    stream=True,
    stream_options={"include_usage": True},
)

4. Run one call and open the trace in the console. The next section shows where the counts appear. The page on exploring traces covers the views in full.

What you get

Token counts appear at several levels of the Traces page, so you can read them per call or per run.

• Trace list. Each row shows a token total and a computed cost for the whole trace.
• Waterfall view. Open a trace and the spans appear as a tree on a timeline. The spans for model calls are badged llm and show their token counts inline. The header shows total tokens and cost for the trace.
• Span detail. Select a model-call span and its Model section lists the provider, requested and response model, input and output tokens, and cost. When the model’s context window is known, it also shows how much of the window was used.
• Context growth. For a trace with several model calls, a strip above the tree plots input tokens per call in order. Click a bar to select that call. This is where a growing prompt shows up first.
• Spans tab. Filter by the LLM kind in the All kinds menu to list every model call. Click the Tokens header to sort by count, which puts your heaviest calls at the top.

A call that reported no token usage shows a blank cost rather than $0, so it never looks free. If a streamed call shows blank tokens, check that include_usage is set.

In TypeScript, streamed chat completions carry token usage when you set include_usage, but no time to first token. Trace such a call as a manual generation if you need that timing. Python records time to first token on streamed calls without extra code.

For the same numbers across your whole workspace, see Monitor AI agents in production.

When something else fits

If you call OpenAI through a framework or router, such as LangChain or LiteLLM, use that integration’s page in the integrations overview. The SDK counts each call once in most setups even when a wrapper sits over the OpenAI call; see Still counted twice for the exceptions. If your OpenAI calls come from a library with no integration, trace them as manual generations and set the usage yourself.

Related