My AutoGen agent suddenly costs more. Why?

Rius shows why your AutoGen agent costs more by pricing every model call and rolling it up per trace, so you can see which part of a run grew. Open an expensive trace and compare its model-call spans with a cheaper run: a different model, a larger prompt, or more calls per run each show up there. To catch the next spike, add a custom alert on cost per trace for the agent.

How it works

AutoGen emits its own spans for agents and tools, and the openai integration records each model call as a ChatCompletion span with its model, token counts and cost. A run looks like this:

handle-ticket                        AGENT  2.8s
├─ create_agent support_agent               0ms
└─ invoke_agent support_agent               2.8s
   ├─ ChatCompletion                 LLM    2.6s
   └─ execute_tool account_status           0ms

Rius prices each model call from the requested model and its token counts, using list prices in US dollars from LiteLLM’s public catalog, then adds the calls up per trace. These are estimates, not billed amounts. A rise in cost per trace therefore comes from one of three things in these spans:

• A different model. The span records the requested model, and each model has its own rates.
• More tokens per call. A longer system message, a longer conversation history or bigger tool results raise the input tokens of every later call.
• More calls per run. More ChatCompletion spans under one invoke_agent span, for example when the agent loops or calls its tool again.

A call whose model has no price, or that reports no token usage, has an empty cost, not $0. A drop in priced coverage can make totals look lower, never higher.

Set it up

1. Make sure the AutoGen integration is on, so model calls arrive with tokens and cost. The AutoGen integration page has the full setup, and Integrations lists how the others attach.

   pip install "glassflow-rius[openai]" "autogen-agentchat" "autogen-ext[openai]"

2. Open Traces and set the time range to cover the jump. The summary strip above the list shows cost and tokens for the traces that match your filters. Try 1d, then 7d, and note when cost per trace moved.

3. Switch to the Spans tab, pick LLM in the All kinds menu, and click the Cost header to sort. Check the model on the costliest calls, and filter by the old and new model to compare token counts.

4. Open one expensive trace in the waterfall. Count the ChatCompletion spans under invoke_agent. For a trace with several model calls, the context growth strip above the tree plots input tokens per call, which is where a growing history shows first. The Map tab has a cost lens that shows where the money went in that run.

5. Select a ChatCompletion span and open Since previous llm call on the Input / Output tab. It marks what each step added to the prompt, so a rewritten system message or a large tool result stands out. A repeated tool call that failed and retried collapses into one row, for example fetch_article · 3 attempts · recovered.

6. Use Compare runs on the trace page to set the expensive run against a cheaper run of the same agent. It shows which steps were added or dropped and where cost moved. The same comparison works across agent versions when your traces carry a version. Details are in Exploring traces.

7. Add a cost alert so the next spike reaches you. In Alerts, create a custom rule scoped to your service:

   FILTERS   Service = my-agent
   RULES     p95(Cost) > $2.00
   WINDOW    1 hour

   Pick a threshold just above your normal p95. Cost is measured per trace over a window of 5 minutes to 24 hours. Add a channel (webhook, email or Slack) so it notifies you.

If the jump began right after you added a manual generation around a call the openai integration already records, that call is counted twice. Keep one of the two.

What you get

• The span that changed: the model, the token counts or the number of model calls per run.
• A cost for each model call and each trace, with the model and tokens beside it.
• A prompt diff between consecutive model calls, so you see what grew.
• A cost alert on p95 cost per trace for your AutoGen service, with notifications to your channel.
• Pricing background in Track LLM and agent cost, and the wider checklist in Monitor AI agents in production.

When something else fits

• If you want an alert on total workspace spend, watch Usage and cost instead. A workspace budget is not an alert metric, but the page shows cost by model and the change against the previous period.
• If your invoice differs from what Rius shows, that is expected: Rius uses list prices and applies no negotiated, batch or long-context discounts.

Related