What does it cost to have an AI SRE watch for a disk filling up every ten minutes?

Tares runs a small triage model on a ten-minute schedule, and a run costs well under a cent. Ten minutes is 144 runs a day, above the default limit of 50 runs per agent, so you raise the limit under Settings, Agents or with TARES_AGENT_DAILY_CAP. The main agent runs only when triage flags something, so the thorough, costlier work happens rarely.

How it works

The pattern is described in Watch, then escalate. It uses two agents linked by a handoff.

• Triage agent. A schedule trigger fires every interval and hands the agent a summary of the window, not raw lines. For each label you name, it shows the count per value against the window before, largest change first, plus a few recent lines. A small model reads that summary and ends every run with either no_op (nothing needs a closer look) or a finding with the verdict investigate.
• Main agent. It has no trigger of its own. It starts only when triage concludes with investigate, and then does the root-cause analysis on that one entity.

So the cost has two parts:

• Triage: 144 runs a day at well under a cent each, which keeps the triage total well under $1.44 a day.
• Main agent: it runs only when triage flags something, at most once per cooldown per entity. With a 30m cooldown that is at most two root-cause runs per hour per entity.

Set it up

1. Label the fields you want counted on the sources that carry your disk data, so the triage agent can see which values moved. Put both agents in one project.
2. Add a schedule trigger that fires every ten minutes. The interval must be at least 1m.

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

3. Create the triage agent on that trigger and seed its prompt from the Triage (on a schedule) preset. Pick the cheapest model on your provider that calls tools reliably.
4. Give the triage agent a budget, so the cadence cannot run up a bill.
5. Raise the daily limit. At ten minutes the agent runs 144 times a day, and the default is 50 runs per agent per day. Change it under Settings → Agents, or set the TARES_AGENT_DAILY_CAP environment variable.
6. Create the main agent from the Root cause after a handoff preset with No trigger: only a handoff starts it. On the triage agent, under When it concludes, add a handoff for the verdict investigate to the main agent with cooldown 30m.

handoffs:
  - { verdict: investigate, agent: rca, cooldown: 30m }   # at most two root-cause runs per hour per entity

The cooldown caps how often the costlier agent can run on the same entity.

What you get

• A predictable bill. Triage costs a small, steady amount per run, and the budget on the agent puts a ceiling on it.
• Most runs are cheap. Most triage runs end as “no finding” with a one-line reason. The runs table shows the reason and the cost of each run.
• Thorough work only when it matters. When triage flags an entity, the main agent fetches its own evidence and writes a root-cause note on the same entity.
• One readable thread. On the project’s Overview, under What the agents found, one card shows the chain: the tick, the triage finding and the root cause, in order.
• A simple off switch. Disable the triage agent or pause its schedule trigger. The watch stops, and with no new investigate findings the main agent goes quiet too. Alert-driven agents keep running as before.

When something else fits

• If your cost target is tight, a longer interval means fewer runs a day. The interval must be at least 1m.
• Alert rules catch what someone wrote a rule for. This watch is for what nobody thought to write a rule for. You can keep both, because turning the watch off leaves alert-driven agents running as before.
• For the alert-driven agent this pattern sits in front of, see AI SRE.

Related