Skip to Content

Where do I see whether the trigger for 5xx errors on an API fired and which run it woke?

Tares shows whether your 5xx trigger fired on the trigger’s own page and under Firings in the console. The run it woke is under Activity and under the agent’s Runs. A trigger evaluates as events arrive, so if it did not fire, check that its sources and filters match the events that came in.

How it works

A trigger watches the sources you list, keyed by a label such as service, and evaluates its condition as events arrive. When the condition holds, it fires and wakes the agent subscribed to it. Tares records both steps, so you can follow the chain in the console:

  • The trigger’s page lists its firings. In the demo from the AI SRE guide, the trigger is named incident and its page is at http://127.0.0.1:8787/triggers/incident. For your own trigger, open it from the console and read its firings.
  • Firings is the console list of firings across triggers.
  • Activity, under Setup, then Advanced setup, shows a firing and the run it woke as one thread. This is the fastest way to tie a firing to its run.
  • Agents, then your agent, then Runs lists every run of that agent, so you can open the one the firing woke.

When the agent finishes, it writes a finding onto the service’s timeline. You see it in Explore for that service, and on the project’s Overview under What the agents found. Open it for the full note and how Tares got there.

In the demo, the incident trigger fires when an alert event arrives (sum(alert_active) > 0 over 1m, per service). A 5xx trigger of your own can react to a fired alert the same way, or threshold the typed fields directly with an error-rate predicate.

Set it up

Use these steps to check a 5xx trigger end to end.

  1. Open the trigger from the console and look at its firings. A firing with a recent time means the condition held.
  2. Open Setup, then Advanced setup, then Activity. Find the firing and read the run it woke in the same thread.
  3. Open Agents, pick the subscribed agent, and open Runs to see the full list of runs and open the one you want.
  4. If the trigger shows no firing, open Explore for the service and confirm the 5xx events are arriving. The metric, the logs and any alert event should sit on one timeline. If they do not appear, the problem is upstream of the trigger, in the source.
  5. If the events are there but the trigger stayed quiet, compare the trigger’s sources and filters with those events. The trigger must list the source the events come from, and each filter must match the event’s fields, such as the service label. Then check the condition, its aggregate, predicate and window, against what arrived.

If you run the bundled demo from the guide, you can cause a 5xx incident on purpose and watch the whole chain. Start Tares first:

tares up

Then inject the fault:

curl -s -XPOST localhost:8080/demo/inject -H 'content-type: application/json' \
  -d '{"scenario": "error_spike"}'

Within about 30 seconds, HighErrorRate fires and the alert lands in Explore next to the error logs. The incident trigger then fires, and incident-first-look wakes. Its firing and run appear in Activity and under the agent’s Runs. To roll back, send {"scenario": "clear"} the same way, and a resolved event lands on the timeline.

What you get

  • A direct answer to whether the trigger fired, from its page or from Firings.
  • The run the firing woke, shown with the firing in one Activity thread, and the complete run history under the agent’s Runs.
  • The agent’s finding on the service timeline and on the project Overview, so you see the conclusion as well as the run.
  • A clear place to look when nothing fired. The trigger evaluates as events arrive, so a missing firing points to the events, the sources the trigger lists, or its filters and condition, and you can check each one in the console.

When something else fits

  • If the trigger fired but the run is not what you expected, open the run from Runs and read the finding. The prompt of your Tares agent is the thing to edit.
  • If events never reach Explore, fix the source first. A trigger cannot fire on events Tares never received.
  • For a first look at how a trigger, an agent and a finding connect, the demo in the AI SRE guide lets you cause a 5xx spike and follow it.

Still stuck? Get help

Last updated on