Skip to Content
Tares

How do I wake an AI SRE once when Sentry issue alerts report high latency?

Tares wakes your AI SRE once per cooldown window when Sentry issue alerts arrive: send them to a webhook source, then add a trigger with a cooldown. Sentry reaches Tares through the webhook connector and a vendor setup, and Tares has no connector of its own for Sentry today. A repeating alert starts one investigation per entity, and the agent receives the entity’s correlated timeline with the dispatch.

How it works

  • Sentry posts each alert delivery to the ingest URL of a Tares webhook source. The payload is stored lossless, whatever its action value.
  • Sentry signs its deliveries. The signature is the credential, so the source needs no Tares API key. You give the source the signature scheme, the signature header and the integration Client Secret.
  • A trigger watches the source and checks its condition as events are ingested. When the condition holds, Tares POSTs one dispatch to each subscriber, and a Tares agent in the project can be woken too.
  • cooldown is the minimum gap between firings for the same entity. While it runs, repeated Sentry deliveries for that entity do not start another investigation.
  • Each dispatch carries the entity’s correlated timeline over context_window, so the agent starts with context and makes no follow-up call. This is the wake-up path described in AI SRE.

The trigger counts the events its source and filters let through. It does not measure latency itself. Scope what reaches the source to your latency alerts: in Sentry, turn on Alert Action for the alert rule that reports high latency and use the integration from that rule. A dedicated source for this integration keeps other Sentry traffic out of the trigger.

Set it up

  1. In Sentry, open Settings, then Developer Settings, and add an integration. Set its Webhook URL to the Tares ingest URL of your source. Subscribe it to the resources you want, such as issues or errors, and turn on Alert Action so alert rules can use it.
  2. Copy the integration Client Secret. You paste it into the Tares source as the signing secret.
  3. Add the webhook source under Sources → Add source → webhook. Declare a label with a const value as the primary label so every Sentry delivery shares one entity. The signing options are described in the Tares release notes, not on the webhook connector page.
- name: sentry-latency connector: webhook config: signature_scheme: hmac_sha256 signature_header: Sentry-Hook-Signature signing_secret: your-client-secret event_type_field: action labels: - { name: service, const: checkout, primary: true }
  1. Save the source and copy its ingest endpoint into the Sentry Webhook URL from step 1.
  2. Check Recent events and Fields on the source page to see which payload fields arrive and what action values they carry.
  3. Add the trigger on the project’s page or in the catalog YAML. The count condition fires on the first delivery, and the cooldown holds back repeats for the same entity:
triggers: - name: sentry_latency sources: [sentry-latency] key_field: service condition: aggregate: count predicate: "> 0" window: 5m group_by: [key_value] emit: kind: sentry_latency context_window: 15m cooldown: 30m
  1. Subscribe an agent. A Tares agent wakes once you turn the trigger on in its project. An external agent calls the subscribe tool with the trigger name and its URL.
  2. To narrow what counts, add a filter such as { field: event_type, op: eq, value: ... } with the action value you saw in Recent events. Filters decide which events count and which are handed over, so check the result before you rely on it.

What you get

  • One investigation per entity per cooldown window, instead of one per repeated Sentry delivery.
  • A dispatch with the entity’s correlated timeline over context_window, ready to hand to the model.
  • Every delivery stored lossless on the source, so you can inspect what Sentry sent under Recent events.
  • Every firing logged, even with zero subscribers. You see firings and per-subscriber delivery status under Firings, on the trigger’s page and on the project’s Activity.
  • Delivery that is at-least-once. Dedupe on dispatch_id in your own endpoint, and acknowledge with a 2xx.

The entity is the primary label of the source. With a const label, one cooldown covers everything this source receives. To get a separate cooldown per service, carry a label that names the service in the payload and make it the primary label.

When something else fits

  • If you want a latency threshold checked inside Tares, send latency metrics from a Prometheus source and use an aggregate such as max(p99_ms) > 1000 in the trigger, instead of relying on Sentry’s alert rule.
  • If you want a periodic summary instead of a reaction to each alert, give the trigger a schedule with condition: {every: 10m}.
  • If a shorter cooldown suits your on-call habits, lower cooldown. A longer one suits alerts that repeat for hours.

Still stuck? Get help

Last updated on