What does it cost to have an AI SRE watch for high latency every ten minutes?

Tares runs a cheap triage agent every ten minutes, which is 144 runs a day, and each run on a small model costs well under a cent. The default limit is 50 runs per agent per day, so raise it under Settings, Agents or with TARES_AGENT_DAILY_CAP. The main root-cause agent runs only when triage flags something, so you pay for thorough analysis only when latency moves.

How it works

The pattern in Watch, then escalate uses two agents linked by a handoff. The triage agent is cheap and frequent. The main agent is thorough and rare.

• Schedule trigger. It fires every interval for all the events of its sources, whether or not anything happened. At every: 10m that is 144 runs a day. The minimum interval is 1 minute.
• A summary, not raw lines. The trigger hands the triage agent a summary of the window. For each label you name, it shows the count per value in this window against the window before, largest change first, plus a few recent lines. A stream of tens of thousands of lines becomes a page the model can read, which keeps each run small.
• Two ways a run ends. Most runs end with no_op: nothing needs a closer look, and the run is a success with no finding. When one entity looks off, the run ends with a finding with verdict investigate, naming the numbers that moved.
• Handoff. An investigate finding hands that entity to the main agent. The main agent is handed the finding, not the logs, and fetches its own evidence. It runs at most once per cooldown per entity. With a cooldown of 30m, that is at most two root-cause runs per hour per entity.

The main agent is not part of the daily cost of watching. It adds cost only on the runs where triage flags something.

Set it up

1. Label the latency-related fields on the source you watch, for example service and status_code on a web_logs source, and put both agents in one project.
2. Add a schedule trigger that fires every ten minutes:

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

3. Create a Tares agent on watch-web and seed its prompt from the Triage (on a schedule) preset. Pick the cheapest model on your provider that calls tools reliably. The preset considers only values that grew at least 3x and by at least 50 events since the window before, and ignores routine noise such as uptime probes. The numbers are in the prompt, so you can tune them by editing it.
4. Set a budget on the triage agent so the cadence cannot run up a bill.
5. Raise the daily run limit. At ten minutes you need 144 runs a day, and the default is 50 per agent. Change it under Settings → Agents, or set TARES_AGENT_DAILY_CAP.
6. Create the main agent in the same project from the Root cause after a handoff preset, with No trigger: only a handoff starts it. On the triage agent, under When it concludes, choose Add a handoff: pick investigate, the main agent and cooldown 30m.

The catalog form of the link looks like this:

agents:
  - name: triage
    trigger: watch-web
    concludes: true
    verdicts:
      - { verdict: investigate, when: one entity looks like a problem and needs a closer look }
    prompt: ...             # from the Triage (on a schedule) preset
    handoffs:
      - { verdict: investigate, agent: rca, cooldown: 30m }   # at most two root-cause runs per hour per entity

The alert-driven agent this sits in front of is described in Build an AI SRE.

What you get

• A predictable base cost. You pay for 144 triage runs a day, each on a small model reading a summary, at well under a cent a run.
• A capped bill. The agent budget and the daily run limit bound what the cadence can spend.
• Thorough analysis only when needed. The main agent runs when triage flags something, at most once per cooldown per entity.
• Visible spend. Open the triage agent’s runs under Agents in Advanced setup. Most rows show “no finding” in the outcome column, each with its reason and cost. A flagged run shows an investigate chip with its headline.
• A full record. The tick, the triage verdict, the handoff and the root-cause note all sit on the entity’s timeline. When triage flags something, the root cause shows on the project Overview under What the agents found.
• An easy off switch. Disable the triage agent or pause its schedule trigger and the watch stops. With no new investigate findings, the main agent goes quiet too.

When something else fits

• If you already have alert rules for latency, alert-driven agents keep running as before, and the watch adds a first level for what nobody wrote a rule for.
• If you want the main agent to wake only on a condition over findings, such as several investigate findings in a window, use a trigger over the findings source instead of a handoff. Use one link or the other, not both, or the main agent runs twice per flag.

Related