How does the triage preset decide that something is worth investigating, and how do I tune it?

Tares decides what is worth investigating in two steps: a schedule trigger summarizes each window, and the Triage preset prompt sets the thresholds. The model considers only values that grew at least 3x and by at least 50 events, or are new with 50 or more, and it ignores routine noise such as uptime probes. The numbers are written in the prompt, so you tune them by editing it.

How it works

The decision has two parts, and each part has its own knobs.

The schedule trigger builds the summary. A schedule trigger fires every interval for all the events of its sources, whether or not anything happened. It does not hand the agent raw lines. For each label you name, it gives the agent:

• the count per value in this window against the window before, largest change first
• a few recent lines

A stream of tens of thousands of lines becomes a page the model can read.

The Triage (on a schedule) preset sets the bar. The preset prompt tells the model to:

• consider only values that grew at least 3x and by at least 50 events since the window before, or are new with 50 or more
• ignore routine noise such as uptime probes
• judge among what is left

Because the thresholds are text in the prompt, the model applies them when it reads the summary. Nothing else in Tares hard-codes them.

Every run ends in a conclusion. The preset ticks Ends every run with a conclusion and lists the verdict investigate. A run ends one of two ways:

• no_op: nothing needs a closer look. The run succeeds with no finding, and the runs table shows “no finding” with a one-line reason.
• finding with verdict investigate: one entity looks off. The finding is recorded on that entity, for example service=checkout, and names the numbers that moved.

The full pattern, including the handoff to a root-cause agent, is in Watch, then escalate.

Set it up

1. Label the fields you want counted. Put the sources to watch in one project and label fields such as service and status_code, so the summary can show which values moved.

2. Add the schedule trigger. On the project’s page click + next to Triggers, then fires: on a schedule. Or add it in the catalog:

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

3. Create the triage agent on that trigger. Seed its prompt from the Triage (on a schedule) preset. Pick the cheapest model on your provider that calls tools reliably, and set a budget.

4. Tune the thresholds in the prompt. Open the agent’s prompt and edit the numbers in the sentence about growth. Examples of what you can change:

   • Raise the 3x factor or the 50-event floor to flag less.
   • Lower them to flag more.
   • Add a line naming other routine noise to ignore.

5. Tune the summary itself. Change every to widen or narrow the window (the minimum is 1m). Change summary_by to count other labels. The model can only judge the labels you list there.

6. Check the verdicts. The preset fills in How it ends with investigate. To pass flagged entities on, add a handoff on the triage agent under When it concludes:

agents:
  - name: triage
    trigger: watch-web
    concludes: true
    verdicts:
      - { verdict: investigate, when: one entity looks like a problem and needs a closer look }
    prompt: ...             # from the Triage (on a schedule) preset
    handoffs:
      - { verdict: investigate, agent: rca, cooldown: 30m }   # at most two root-cause runs per hour per entity

7. Mind the daily limit. At a 10 minute interval the agent runs 144 times a day, above the default limit of 50 runs per agent per day. Raise it under Settings → Agents or with TARES_AGENT_DAILY_CAP.

What you get

• A first level that says “normal” in a line most of the time and flags one entity only when it clears the bar you set.
• Findings that name the numbers that moved, recorded on the entity they concern.
• Thresholds you can read and change in plain text, with no code to deploy.
• A low cost. A small model reading a summary costs well under a cent a run, and the root-cause agent runs only when triage flags something, at most once per cooldown per entity.
• A record of every step. Each tick, triage verdict and handoff sits on the entity’s timeline. In the triage agent’s runs, most rows show “no finding” with a reason and cost, and a flagged run shows an investigate chip with its headline.

To stop the watch, disable the triage agent or pause its schedule trigger.

When something else fits

• If you already know the condition you care about, such as a count crossing a number in a window, a condition trigger fits better than a schedule summary. See Build an AI SRE for the alert-driven agent this pattern sits in front of.
• If you need a rule across several investigate findings or several agents, wake the main agent with a trigger over findings instead of a handoff. Use one link or the other, not both.

Related