Which model should triage and which should root-cause a disk filling up?

Tares lets you run the cheapest tool-calling model on the frequent triage agent and a stronger hosted or 30B-class-and-above model on the rare root-cause agent. You pick each model on that agent’s own form. Triage reads a summary every few minutes and mostly says “normal”. The root-cause agent runs only when triage flags an entity, so its higher price per run is paid rarely.

How it works

Two agents work as a pair, linked by a handoff. This is the pattern in Watch, then escalate, and it sits in front of the alert-driven agent described in AI SRE.

• Triage agent. A schedule trigger wakes it every interval. It gets a summary of the window, not raw lines: for each label you name, the count per value against the window before, largest change first, plus a few recent lines. It ends every run with a conclusion, either no_op (nothing to look at) or a finding with the verdict investigate.
• Root-cause agent. It has no trigger and never wakes on its own. A handoff from triage starts it on the same entity. It receives the triage finding, not the logs, and fetches its own evidence with read, stats and any MCP tools you give it. It then writes its note with conclude, using the verdict rca or resolved.

Because the two agents have different jobs, they have different model needs:

• Triage runs often, so cost per run matters most. The guide says to pick the cheapest model on your provider that calls tools reliably. A small model reading a summary costs well under a cent a run.
• Root cause runs rarely, at most once per cooldown per entity, and it works on real data. For root-cause work on real data, use a hosted model or a 30B-class one and above. A small 8B-class model can invent events on an empty result, so treat that size as a way to try things out.

Each agent form has a provider picker next to a model picker, so the two agents can sit on different models, or even different providers.

Set it up

1. Add the providers you need under Settings → Model providers. An instance can hold several: Anthropic, OpenAI, or an OpenAI-compatible endpoint such as LiteLLM, OpenRouter, Ollama or vLLM. Saving a provider reads the models it serves, so the agent form offers exactly what your key can use. Use Refresh models if the list changes later. See Model providers for the kinds and what you supply.
2. Pick the sources to watch and label the fields you want counted. For a disk-related stream, label the values you want to see move, so triage can tell which one grew.
3. Add a schedule trigger on the project. In the catalog it looks like this:

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

4. Create the triage agent on that trigger and seed its prompt from the Triage (on a schedule) preset. On its form, pick the cheapest model that calls tools reliably, and set a budget so the cadence cannot run up a bill.
5. Create the root-cause agent in the same project, seed its prompt from Root cause after a handoff, and set its trigger to No trigger: only a handoff starts it. On its form, pick a hosted model or a 30B-class one and above.
6. Add the handoff on the triage agent under When it concludes. In the catalog, the provider and model fields sit on each agent:

agents:
  - name: triage
    trigger: watch-web
    concludes: true
    provider: anthropic
    model: ...              # your cheapest tool-calling model
    verdicts:
      - { verdict: investigate, when: one entity looks like a problem and needs a closer look }
    prompt: ...             # from the Triage (on a schedule) preset
    handoffs:
      - { verdict: investigate, agent: rca, cooldown: 30m }   # at most two root-cause runs per hour per entity
  - name: rca               # no trigger: it runs only when handed off
    concludes: true
    provider: anthropic
    model: ...              # your stronger model
    verdicts:
      - { verdict: rca, when: you found what is failing and why }
      - { verdict: resolved, when: it was noise, or it is already over }
    prompt: ...             # from the Root cause after a handoff preset

7. Raise the daily run limit. A 10 minute interval is 144 triage runs a day, above the default of 50 runs per agent per day. Change it under Settings → Agents or with TARES_AGENT_DAILY_CAP.

If you leave the model empty on an agent, it runs on that provider’s default model. If you leave the provider on the cell default, it follows the instance default.

What you get

• A triage agent that costs little per run and answers “normal” in one line most of the time, with the reason shown in the runs table as “no finding”.
• A stronger model spent only on the entity that moved, so the expensive reasoning runs at most once per cooldown per entity.
• One card under What the agents found on the project Overview for the whole chain: the tick, the triage finding and the root cause, in order.
• A cost record you can read. Every run records the provider and model it ran on, and the spend meter on the Overview prices each run from the model that answered. Once more than one provider has run, spend is split by provider. A model with no price, such as a local Ollama model, records its tokens with no dollar figure, and the meter counts those calls as unpriced.
• An easy way to change your mind. Swap the model on either agent’s form and the next run uses it, with the history still readable.

When something else fits

• If one model on both agents is simpler to run and your volume is low, leave both on the default provider. You can split them later without changing the handoff.
• If you already have an alert rule for the disk, an alert-driven agent covers it directly, and the triage pair is for what nobody wrote a rule for. See Model providers if you need a local small model for triage and check that it calls tools reliably first.

Related