Why does my OpenAI Agents SDK agent make more LLM calls than usual?
Rius records every model call your OpenAI Agents SDK agent makes as its own LLM span, so you can count the calls in a trace and see what happened before each extra one. Open a slow or costly trace, count the LLM spans, and check the tool spans, the input token growth and any retries between them. A custom alert on the LLM calls metric tells you the next time the count jumps.
How it works
The OpenAI Agents SDK integration captures model calls through the openai integration underneath the framework. What you see in a trace depends on three things:
- Model calls. Each one is an LLM generation with model, tokens, cost and time to first token. The Responses API calls are all named
Response. Only spans badged LLM count as LLM calls. Agent, tool, retrieval and embedding spans never do. - Run structure. Your
observewrapper aroundRunner.run()is the root AGENT span. Each function tool you wrap with the TOOL kind is a TOOL span. - Handoffs. A handoff has no span of its own. It shows only as consecutive generations that belong to different agents.
An extra model call therefore appears as an extra Response span in the tree. The spans around it show what triggered it. More tool spans before the extra calls point to a tool loop. A rising input token count points to a growing prompt. Failed attempts point to retries. A second agent’s generations point to a handoff.
Set it up
- Make sure the run and its tools are wrapped, so the trace shows what sits between the model calls. This is the setup from the integration page. A tool wrapped with the TOOL kind is also the only kind the Tool Loop alert can count. A tool traced with a bare
observeis a CHAIN span and is never counted.
import rius
from agents import Agent, Runner, function_tool, set_tracing_disabled
from rius import SpanKind
rius.init(api_key="ri_...", service_name="my-agent")
set_tracing_disabled(True)
def lookup(email: str) -> str:
return "active" # replace with your account lookup
@function_tool
@rius.observe(kind=SpanKind.TOOL)
def account_status(email: str) -> str:
"""Fetch a customer's account standing by email."""
return lookup(email)
support_agent = Agent(
name="support_agent",
instructions="Answer support questions; check accounts with the tool.",
tools=[account_status],
model="gpt-5-mini",
)
@rius.observe(name="handle-ticket", kind=SpanKind.AGENT)
def handle_ticket(ticket: str) -> str:
# max_turns caps the agent loop; past it the run raises MaxTurnsExceeded
result = Runner.run_sync(support_agent, ticket, max_turns=10)
return str(result.final_output)
with rius.session("ticket-1093"):
print(handle_ticket("Why is my account on hold?"))The framework’s own cap is max_turns on Runner.run_sync (or Runner.run). A run that hits it raises MaxTurnsExceeded, which the root span records as an error.
-
Open Traces in the console and pick a run that made too many calls. The header shows span count, total tokens and cost. In the waterfall, count the
Responsespans and read what sits between them. -
Look at the context growth strip above the tree. It plots input tokens per LLM call in order, which is where an unbounded history shows up first. Click a bar to select that call.
-
Select an LLM span, open Input / Output and switch on Since previous llm call. Each message is marked added, removed or changed, so you see what each step added to the prompt, such as a tool result that keeps being appended.
-
Check for retries. Back-to-back calls of the same tool with the same input collapse into one row such as
fetch_article · 3 attempts · recovered, when every attempt but the last failed. -
Use Compare runs on the open trace to put it next to a normal run of the same agent. You see which steps were added or dropped and where cost or latency moved.
-
To see the pattern across many runs, open the Spans tab and choose LLM in the All kinds menu. Sort by Tokens or Cost, or paste a session ID in the search box to list every span in that session.
-
Create a custom alert so the next spike reaches you. Rules measure only the spans your filters match, and the LLM calls metric counts LLM spans. Filter by Service or Agent. The threshold and window below are examples, so pick values from your normal traffic:
FILTERS Service = my-agent
RULES count(LLM calls) > 200
WINDOW 15 minutesTo alert on cost per run, use avg(Cost) with the same filter. It averages cost per trace, not the workspace total.
What you get
- A count of model calls per run, with the tool spans, retries and prompt changes next to each call.
- A per-call input token plot that shows whether calls are multiplying or each call is simply getting bigger.
- A side-by-side comparison with a normal run, plus the span list for the whole window.
- An alert on LLM call volume or cost per trace, delivered through your alert channels.
- For the exact trace and span controls, see Exploring traces. For the wider monitoring picture, see Monitor AI agents in production.
Costs shown are list-price estimates in US dollars, not billed amounts.
When something else fits
- If you need to see where one agent hands off to another, add an
observewrapper per agent entry, or use the framework’sRunHooksto open spans at agent boundaries. Without them, the genericResponsenames do not tell you which agent spoke. - Rius shows what the agent did and what it cost. It does not offer evals or scores, so judging whether the extra calls improved answers needs another process.
- For other frameworks and clients, see the integrations overview.