Do I need a separate triage agent for high latency, or can one agent do it all?

Tares lets you start with one agent, and a separate cheap triage agent pays off when you check latency often and only a few checks need a deep look. One agent is simpler to run. A triage and main pair keeps the strong model for the rare cases, which saves model spend when checks are frequent. The Watch, then escalate guide shows the pair step by step.

How it works

The choice comes down to how often the agent wakes up.

• One agent. A single agent does the whole job: it looks at the evidence and writes the root cause. This is the shape of the alert-driven agent in AI SRE. A rule fires only when something is wrong, so the agent runs rarely and a strong model on every run is fine.
• Two agents. A schedule trigger wakes a triage agent every few minutes. Each tick hands it a summary of the window instead of raw lines: for each label you name, the count per value against the window before, largest change first, plus a few recent lines. Most of the time the triage agent ends with no finding and a one-line reason. When one entity moves, it ends with the verdict investigate, and a handoff starts the main agent on that same entity.

The two agents split the work by cost. The triage agent is cheap and frequent. The main agent is thorough and rare. It receives the triage finding, not the logs, and fetches its own evidence with read, stats and any MCP tools you give it. It then writes its root-cause note on the same entity.

Every step lands on the entity’s timeline: each tick, each triage verdict, each handoff and each root-cause note.

Set it up

To run the pair for a high-latency watch, follow these steps. If you pick a single agent, skip to step 1 and stop after the first agent.

1. Put the sources you want to watch and both agents in one project. Label the fields you want counted, such as service and status_code, so the triage agent can see which values moved. The guide’s example uses request logs in a web_logs source.
2. Add a schedule trigger. It fires every interval whether or not anything happened. The minimum interval is 1m.

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

3. Create the triage agent on watch-web and seed its prompt from the Triage (on a schedule) preset. Choose the cheapest model on your provider that calls tools reliably, and set a budget so the cadence cannot run up a bill. The preset considers only values that grew at least 3x and by at least 50 events since the window before, and you tune those numbers by editing the prompt.
4. Create the main agent from the Root cause after a handoff preset, with no trigger, so only a handoff starts it. Then add the handoff on the triage agent under When it concludes.

agents:
  - name: triage
    trigger: watch-web
    concludes: true
    verdicts:
      - { verdict: investigate, when: one entity looks like a problem and needs a closer look }
    prompt: ...             # from the Triage (on a schedule) preset
    handoffs:
      - { verdict: investigate, agent: rca, cooldown: 30m }   # at most two root-cause runs per hour per entity
  - name: rca               # no trigger: it runs only when handed off
    concludes: true
    verdicts:
      - { verdict: rca, when: you found what is failing and why }
      - { verdict: resolved, when: it was noise, or it is already over }
    prompt: ...             # from the Root cause after a handoff preset

5. Raise the daily run limit. A 10 minute schedule is 144 runs a day, and the default limit is 50 runs per agent per day. Change it under Settings → Agents or with TARES_AGENT_DAILY_CAP.

Use either the handoff or a trigger over findings to wake the main agent, not both, or it runs twice per flag. For the trigger option, see Watch, then escalate.

What you get

• Frequent latency checks at a low price. A small model reading a summary costs well under a cent a run.
• A strong model that runs only when triage flags something, at most once per cooldown per entity. With a 30 minute cooldown that is at most two root-cause runs per hour per entity.
• One card on the project Overview under What the agents found for the whole chain, with the headline and the next step. Click it to see the tick, the triage finding and the root cause in order.
• A runs table for the triage agent that shows mostly “no finding” rows, each with its reason and cost, and an investigate chip on a flagged run.
• A simple off switch. Disable the triage agent or pause its schedule trigger, and the main agent goes quiet too. Alert-driven agents keep running as before.

When something else fits

• Choose one agent when an alert rule starts the investigation. It runs only when something is wrong, so a second agent adds setup without saving much.
• Choose one agent when you check rarely, because the cost gap between the two shapes only shows at a high check rate.
• Choose the pair when the schedule runs every few minutes and most ticks end in “normal”.

Related