My AutoGen agent is slow to respond. Why?

Rius shows where an AutoGen run spends its time by recording each agent, model call and tool call as a timed span, so you can find the slow step in the trace waterfall. Open the slowest trace and read the durations on the invoke_agent, ChatCompletion and execute_tool spans to see whether the time goes to the model, a tool or a gap between them. Then add a duration alert so the next slow run reaches you first. The Monitor AI agents in production guide covers the wider workflow.

How it works

With the AutoGen integration, the framework’s own OpenTelemetry spans and the openai integration produce one trace per unit of work. Your observe wrapper on the entrypoint is the root, and each step below it carries its own duration:

handle-ticket                        AGENT  2.8s
├─ create_agent support_agent               0ms
└─ invoke_agent support_agent               2.8s
   ├─ ChatCompletion                 LLM    2.6s
   └─ execute_tool account_status           0ms

This is the example trace from the integration page. Read it like this:

• A long ChatCompletion span. The model call is the slow step. The span shows the model, input and output tokens and cost, so you can check whether the prompt grew.
• A long execute_tool span. The tool itself is slow. The span is named after the tool.
• A long invoke_agent span with short children. The time sits between steps, in your own code or in the agent’s loop.
• A tool that runs as its own single-span trace. AutoGen runs tools in a detached context, so an observe wrapper on a tool function splits off from the run. Remove the wrapper, because the execute_tool spans already cover tools.

For streamed calls, Rius also computes time to first token, the wait before the first chunk arrives. Only streamed calls record it, and a workspace whose calls never stream shows a dash.

Set it up

1. Make sure the integration is in place. This is the install line from the AutoGen integration page. Integrations lists the other frameworks and libraries.

   pip install "glassflow-rius[openai]" "autogen-agentchat" "autogen-ext[openai]"

2. Open Traces in Exploring traces. Set the time range to cover the slow period and read the latency column. The strip above the list shows latency percentiles for the traces that match your filters. Filter by the agent so you compare like with like.

3. Open the slowest trace. The waterfall puts the spans on a timeline, badges them by kind and shows token counts on model calls. Find the widest span, then open the Map tab and pick the latency lens to see the hot spot as a graph. Clicking a node lands you on that node’s spans in the tree.

4. To compare many runs at once, use the Spans tab. Choose LLM in the All kinds menu and click the Duration header to sort. For tools, type execute_tool or the tool’s name in the search box, which matches span names, and sort the same way.

5. Check whether the model is slow because the prompt keeps growing. On a trace with several model calls, the context growth strip above the tree plots input tokens per call in order. On a model call span, Since previous llm call on the Input / Output tab shows what each step added to the prompt.

6. Make a hung run visible. A run reads as running… only when partial spans are on. Set this where the agent runs, then filter the list on incomplete to find runs that started and never ended:

   export RIUS_PARTIAL_SPANS=true

7. Create a custom alert on duration or time to first token, with channels on webhook, email or Slack. To alert on whole runs, filter by Service or Agent. To alert on model calls only, set Span kind to LLM, because a rule measures only the spans it matches.

What you get

• The slow step named: the model call, a tool or the agent’s own loop, with its duration in the waterfall.
• A comparison across runs on the Spans tab, sorted by duration, instead of opening traces one at a time.
• Signs of a growing prompt, from input tokens per call and the diff against the previous call.
• Retry loops shown as one row. Back-to-back calls of the same tool or step with the same input collapse into a line such as fetch_article · 3 attempts · recovered when every attempt but the last failed.
• A trend view on Home: latency p50, p95 and p99 for finished traces, plus time to first token for streamed calls, over 15 minutes to 30 days. See Usage and cost.
• An alert on duration or time to first token that tells you before a user does.

When something else fits

• If the slow runs never finish, the latency percentiles leave them out until they end. Turn on partial spans and filter for running… traces instead.
• If the slowness sits inside one provider call, such as a long wait before the first token, look at time to first token for that model on Home. It covers streamed calls only.

Related