What does it cost to have an AI SRE watch for crash loops every ten minutes?

Tares runs a cheap triage agent every ten minutes, which is 144 runs a day, and a small model reading a summary costs well under a cent a run. The main root-cause agent runs only when triage flags something. The default limit is 50 runs per agent per day, so raise it under Settings, Agents or with TARES_AGENT_DAILY_CAP.

How it works

Watching for crash loops uses two agents in one project, linked by a handoff. This is the pattern in Watch, then escalate, and it sits in front of the alert-driven agent described in AI SRE.

• Triage agent. A schedule trigger fires every interval for all the events of its sources, whether or not anything happened. It hands the agent a summary of the window, not raw lines. For each label you name, the summary gives the count per value in this window against the window before, largest change first, plus a few recent lines. A stream of tens of thousands of lines becomes a page the model can read.
• Main agent. It has no trigger of its own. It starts only when triage ends a run with the verdict investigate, and then it takes over on that same entity.

The cost follows from that split:

• Triage runs on every tick. At ten minutes that is 144 runs a day. A small model reading a summary costs well under a cent a run, so 144 runs add up to well under 1.44 dollars a day.
• The main agent costs something only when triage flags an entity. It runs at most once per cooldown per entity. With the cooldown of 30m in the example, that is at most two root-cause runs per hour per entity.

Most triage runs end with no_op. The run counts as a success with no finding, and the runs table shows it as “no finding” with a one-line reason and its cost.

Set it up

1. Label the fields you want counted on the sources that carry your crash signals, such as service and status_code, and put both agents in one project.
2. Add a schedule trigger at ten minutes. On the project’s page click + next to Triggers, then fires: on a schedule. Or in the catalog:

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

3. Create the triage agent on that trigger and seed its prompt from the Triage (on a schedule) preset. Pick the cheapest model on your provider that calls tools reliably.
4. Set a budget on the triage agent, so the cadence cannot run up a bill.
5. Raise the daily run limit. Ten minutes is 144 runs a day and the default limit is 50 runs per agent per day. Change it under Settings → Agents, or set the TARES_AGENT_DAILY_CAP environment variable for Tares.
6. Create the main agent in the same project, seed its prompt from Root cause after a handoff, and set its trigger to No trigger: only a handoff starts it. On the triage agent, under When it concludes, choose Add a handoff. Pick investigate, the main agent and a cooldown of 30m.

The Watch, then escalate guide has the full catalog YAML for both agents and the handoff.

What you get

• A predictable bill. Triage is the only cost that scales with the clock, it uses a small model, and its budget caps the spend.
• 144 checks a day for crash loops, with a one-line reason on every quiet run.
• Thorough root-cause work only when it is needed. The main agent fetches its own evidence with read, stats and any MCP tools you give it, then writes its note on the same entity.
• A record of every step. Each tick, triage verdict, handoff and root-cause note sits on the entity’s timeline, and a flagged chain shows as one card under What the agents found on the project’s Overview.
• A way to stop it. Disable the triage agent or pause its schedule trigger. With no new investigate findings, the main agent goes quiet too.

When something else fits

• If you already know the exact condition that signals a crash loop, an alert rule or a condition trigger fires only when it is met, so it needs no scheduled runs.
• If ten minutes is more often than you need, a longer interval means fewer runs a day. The schedule can be as short as 1m, which is the minimum.

Related