My Pydantic AI agent makes more LLM calls than usual. Why?
Rius shows every model request your Pydantic AI agent makes as its own chat span under the run’s invoke_agent span, so you can count the calls and see what triggered each extra one. Open a trace with a high call count and read its steps in order: each tool call is an execute_tool span between two chat spans. To catch it next time, add a custom alert on the LLM calls metric, scoped to your agent. Rius records what happened in the run, so the spans show where the extra calls came from, not why the model chose them.
How it works
With Agent.instrument_all(True), Pydantic AI sends its own OpenTelemetry spans through Rius, and Rius reads them directly:
- An agent run is the root span,
invoke_agentfollowed by the agent’s name. - A model request is a generation span,
chatfollowed by the model name, with tokens and cost. - A tool call is a span named
execute_toolfollowed by the tool’s name.
A run that calls one tool looks like this:
invoke_agent support_agent 6.0s
├─ chat gpt-5-mini 3.3s decides to call the tool
├─ execute_tool account_status 0ms
└─ chat gpt-5-mini 2.7s answersIn Rius, an LLM call is a span the trace view badges as LLM. Agent and tool spans never count, so the number of LLM calls in a trace is the number of chat spans. When a run makes more calls than usual, you see more chat spans under the same invoke_agent root, usually with more execute_tool spans between them. Each extra chat span carries its own tokens and a list-price cost estimate in US dollars, so the same trace shows what the extra calls cost.
The pattern in the tree points to the cause. Repeated execute_tool spans for the same tool mean the agent keeps calling it. A trace with many chat calls and few tool calls points to something else, such as a run that was retried.
Set it up
-
Make sure Pydantic AI’s spans reach Rius and each conversation is one session. Setup is covered in Pydantic AI, and the other frameworks are listed under Integrations.
import rius from pydantic_ai import Agent from pydantic_ai.usage import UsageLimits rius.init(api_key="ri_...", service_name="my-agent") Agent.instrument_all(True) support_agent = Agent( "openai:gpt-5-mini", name="support_agent", instructions="Answer support questions; check accounts with the tool.", ) with rius.session("ticket-1093"): result = support_agent.run_sync( "Why is my account on hold?", # caps model requests per run; past it the run raises UsageLimitExceeded usage_limits=UsageLimits(request_limit=10), ) print(result.output) -
Open Traces and filter to your agent. Compare the span count and token total of a heavy run with a normal one. Switch to the Spans tab, pick LLM in the All kinds menu, and paste a session ID into the search box to list every model call in that conversation. The page is described in Exploring traces.
-
Open the heavy trace and read the waterfall. Count the
chatspans underinvoke_agent, and look for the sameexecute_toolspan repeating. Back-to-back calls of the same tool with the same input collapse into one row such asfetch_article · 3 attempts · recoveredwhen every attempt but the last failed. Press]lto jump between LLM spans and]eto jump between failed spans. -
Check the Context growth strip above the tree. It plots input tokens for each LLM call in order, so an unbounded history shows up as bars that keep rising.
-
Select a later
chatspan and open Since previous llm call on the Input / Output tab. It marks each message as added, removed or changed since the previous call, so you see what each step added to the prompt. This needs captured content, which Pydantic AI controls through itsInstrumentationSettings. -
Use Compare runs on the open trace to put it next to another run of the same agent. Rius shows which steps were added or dropped and where cost or latency moved.
-
Add a custom alert so you hear about it next time. Filter Agent to your agent, or Service to measure whole runs, add the rule
count(LLM calls) > N, and set the window. Choose N from what a normal window looks like for your agent. Attach a webhook, email or Slack channel. See Custom alerts.
What you get
- The exact number of model calls in any run, with each call’s model, tokens and cost on its own
chatspan. - The sequence of tool calls and model calls, so you can tell a repeated tool from a longer conversation or a retried run.
- A view of how the prompt grew between calls, from the context growth strip and the diff against the previous LLM call.
- A side-by-side comparison of a heavy run and a normal one from the same agent.
- An alert on LLM call volume for your agent or service, with the observed value recorded on each firing. Use it with a Context usage rule, for example
p95(Context usage) > 0.8, if you also want to know when calls fill most of the model’s window.
For more on reading traces across all your agents, see Monitor AI agents in production.
When something else fits
- If the spans show a normal loop but the answers are wrong, you need to judge answer quality. Rius does not run evals or score outputs, so use your own tests for that.
- If a rule covers only tool, chain or agent spans, it measures no tokens or cost. To alert on model spend, filter by Model, or set Span kind to LLM.
- To change how many requests the agent makes, adjust the agent’s own code and its Pydantic AI configuration, such as the
request_limitinUsageLimits, then use Rius to confirm the call count dropped.