How do I escalate slow database queries from a cheap triage agent to a root-cause agent?

Tares escalates slow database queries with a handoff on the triage agent: when its run concludes with the verdict investigate, the root-cause agent starts on that one entity. The handoff has its own cooldown, so the thorough agent runs only when triage asks for it, and at most once per cooldown per entity. The root-cause agent needs no trigger of its own.

How it works

Two agents sit in one project, linked by a handoff made of three values: a verdict, the agent that takes over, and a cooldown.

• The triage agent wakes on a schedule trigger. That trigger gives it a summary of the window: for each label you name, the count per value against the window before, largest change first, plus a few recent lines. A large stream of slow query events becomes a page the model can read.
• Triage ends every run with the conclude tool. If nothing needs a closer look, the run is a success with no finding. If one entity looks off, such as one database or service, it records a finding with the verdict investigate on that entity.
• Tares matches the verdict (case does not matter) and starts the root-cause agent on the same entity. The agent is handed the finding, not the raw data, and fetches its own evidence with its tools.
• The cooldown caps handoffs from triage to that agent at one per entity within the window. A handoff inside the cooldown starts no run, and the results say handoff to rca skipped: cooldown.

The root-cause agent runs whether or not it is on for its own trigger, so you give it no trigger. The run is recorded with woken_by: handoff and the triage run as its parent, so the triage finding and the root cause read as one thread. The Hand off to another agent page covers every rule, and the Watch, then escalate guide walks through the whole pattern.

Set it up

1. Put the source that carries your slow query events in a project, with labels for the fields you want counted, such as the database or service name. Both agents live in this project.
2. Add a schedule trigger on that source. It fires every interval whether or not anything happened, and summary_by names the labels to count.

triggers:
  - name: watch-web
    sources: [web_logs]
    key_field: service
    condition:
      every: 10m                          # at least 1m
      summary_by: [service, status_code]  # labels to count; default: the trigger's key field

Swap web_logs and the labels for the source and labels that hold your slow queries. 3. Create the triage agent on that trigger. Seed its prompt from the Triage (on a schedule) preset, pick the cheapest model on your provider that calls tools reliably, and set a budget. The preset ticks Ends every run with a conclusion and lists the verdict investigate. 4. Create the root-cause agent in the same project. Seed its prompt from Root cause after a handoff, which comes with the verdicts rca and resolved, and set its trigger to No trigger: only a handoff starts it. 5. Open the triage agent, and under When it concludes choose Add a handoff. Pick investigate, the root-cause agent and a cooldown. The default is 30m.

agents:
  - name: triage
    trigger: watch-web
    concludes: true
    verdicts:
      - { verdict: investigate, when: one entity looks like a problem and needs a closer look }
    handoffs:
      - { verdict: investigate, agent: rca, cooldown: 30m }
  - name: rca               # no trigger: it runs only when handed off

6. Raise the daily run limit if the cadence needs it. A 10 minute schedule is 144 runs a day, above the default of 50 runs per agent per day. Raise it under Settings → Agents or with TARES_AGENT_DAILY_CAP.

What you get

• A frequent, low-cost check that says “normal” in a line on most ticks and spends nothing on deep analysis.
• A thorough root-cause run only for the one entity triage flags, at most once per cooldown. With 30m that is two runs per hour per entity.
• A skipped handoff is never silent. If the target is already running for that entity, or has hit its daily cap or budget, the handoff is recorded as a capped run with the reason.
• One thread on the project’s Activity: the tick, the triage finding and the root cause in order. On the Overview, one card under What the agents found shows the headline and the next step.
• A chain limit. An agent that was handed off to can hand off in turn, and after three handoffs in a row the next is recorded as a capped run with the error handoff chain stopped at depth 3.

To stop the watch, disable the triage agent or pause its schedule trigger. With no new investigate findings, the root-cause agent goes quiet too.

When something else fits

• If you need a condition before escalating, such as several investigate findings in a window or findings from more than one agent, wake the root-cause agent with a trigger over the findings source instead. Use that link or the handoff, not both, or the agent runs twice per flag.
• If you already know the exact rule that defines a problem query, an alert-driven agent from AI SRE fits better than a scheduled triage agent.

Related