How do I route CloudWatch alarms to an AI SRE?

Tares receives CloudWatch alarms through the webhook connector and a vendor setup in Amazon EventBridge, because Tares has no connector of its own for CloudWatch today. An EventBridge rule sends each alarm state change to a webhook source’s ingest URL. A trigger then wakes your agent with the alarm on one correlated timeline next to your other signals.

How it works

CloudWatch alarms reach Tares through EventBridge, not through a CloudWatch-specific connector:

• EventBridge matches alarm events on the default event bus (source aws.cloudwatch, detail-type CloudWatch Alarm State Change) and posts each one as a JSON body to an API destination.
• The API destination points at the ingest URL of a webhook source in Tares. The webhook connector parses the body whatever the Content-Type and stores the payload lossless.
• source and detail-type are top-level fields. detail.alarmName works as a one-level dotted label. detail.state.value is two levels deep, so flatten it with an input transformer on the rule target if you want it as a label or in the text template.
• A trigger watches the source. When it fires, it wakes the agent you chose, and the agent reads the correlated timeline for that entity, as described in Build an AI SRE.

CloudWatch alarm actions that publish to SNS are a different route and do not fit. An SNS HTTPS subscription cannot send a custom header, and its SubscriptionConfirmation message would never be confirmed. Use EventBridge with an API destination instead.

Set it up

1. Create the webhook source in Tares. In the console go to Sources, Add source, webhook. Declare a label for each payload field you want to correlate on, and mark one as primary. Save, then copy the ingest endpoint from the source page. It has the form /ingest/ followed by the ingest key.

- name: cloudwatch-alarms
  connector: webhook
  config:
    event_type: cloudwatch_alarm
    text_template: '{source} in {region}'
    labels:
      - { name: alarm, field: detail.alarmName, primary: true }
      - { name: region, field: region }

2. Create an EventBridge API destination. Set its endpoint to the Tares ingest URL, which must be public HTTPS, and its method to POST.

3. Give it a connection. EventBridge requires one on every API destination. Choose API key authorization with the header name Authorization and the value Bearer followed by an ingest-scoped API key. The key only matters when Tares has auth on.

4. Create a rule on the default event bus. Match source aws.cloudwatch and detail-type CloudWatch Alarm State Change, and set the API destination as the rule’s target. If you want the alarm state as a label or in the text template, add an input transformer on the target that flattens detail.state.value to a top-level field.

5. Add a trigger that wakes the agent. This one fires when an alarm event arrives:

- name: cloudwatch-alarm
  sources: [cloudwatch-alarms]
  filters:
    - { field: event_type, op: eq, value: cloudwatch_alarm }
  condition: { aggregate: count, predicate: "> 0", window: 5m }
  cooldown: 5m

6. Choose who gets woken. Point the trigger at a Tares agent, your own agent service over a webhook, or a Slack channel, as the AI SRE guide describes.

7. Verify. Use test_source with a sample payload to check the label mapping first. Then trigger an alarm state change and check Recent events on the source page. The Fields view shows which payload fields arrive and how often, which helps you pick a reliable key.

What you get

• Every alarm state change you match lands as an event on a timeline in Tares, with the full payload stored lossless.
• The alarm sits on one clock with your metrics, logs and deploys, as long as the sources share a label such as the entity name. The woken agent starts from that correlated read instead of from the alarm text alone.
• Your agent is woken by the trigger, so nobody has to poll and nobody has to page it. The cooldown on the trigger limits repeat firings.
• You keep CloudWatch alarms where they are. Tares ingests what fired and handles the correlation and waking.

When something else fits

• If your alerts already come from Prometheus or Alertmanager, use the prometheus_alerts or alertmanager connectors, which Tares has connectors for, and skip EventBridge.
• If the signal is OpenTelemetry telemetry rather than alarms, send it over OTLP. If it is data you can read from an API, http_poll against that API is an alternative.
• If you can only use SNS delivery for alarms, this route will not work. Move the delivery to an EventBridge rule with an API destination.

Related