How can an AI SRE see GitHub commits and pull requests just before CloudWatch alarms?
Tares puts GitHub commits and pull requests and CloudWatch alarms on one timeline per service, so an AI SRE reads the change and the first alarm in order. GitHub connects through its own connector. CloudWatch reaches Tares through the webhook connector and an EventBridge setup. The agent compares the change time with the first alarm time and writes the gap into its finding.
How it works
Sources that carry the same label, such as service, merge into one timeline on one clock. That is the read an agent works from in an AI SRE setup, and the order of events on it is what you want to see.
- GitHub. The GitHub connector delivers pull requests, pushes, reviews, releases and CI runs as they happen when you use the GitHub App. With a personal token it polls every 2 minutes and reports commits and pull requests opened, merged or closed. Both ways produce the same events, with labels such as
repo,action,author,branchandsha. - CloudWatch. Tares has no connector of its own for CloudWatch today. An EventBridge rule posts each alarm state change to a webhook source, which stores the payload lossless. The payload has a top-level
timefield, andevent_time_fieldpoints the source at it, so the alarm sits on the timeline at its own event time. - The read. The agent sees the merged pull request or push, then the first
ALARMevent, in sequence. The gap between them is the evidence it reasons from. A token source can lag a commit by up to its poll interval, so use the App when the gap is short. - The write-back. The agent records a finding on the service’s timeline with what broke, since when and the likely cause tied to evidence lines. Put the change time, the first alarm time and the gap in that note, and the next investigation starts from it.
Set it up
- Connect GitHub. On Tares Cloud choose Connect GitHub in your workspace settings. On a self-hosted Tares open Settings > GitHub and create the App, or add a personal token and one source per repository. A token source looks like this, with a
servicelabel added so it shares a key with your alarms:
- name: checkout-api
connector: github
poll: 2m
config:
repo: acme/checkout-api
credential: github
labels:
- { name: service, const: checkout, primary: true }
- { name: repo, field: repo }- Create a webhook source for the alarms. Use the same
servicelabel, read the event time fromtime, and add the alarm name as a one-level dotted label:
- name: cloudwatch-alarms
connector: webhook
config:
event_type: cloudwatch_alarm
event_time_field: time
labels:
- { name: service, const: checkout, primary: true }
- { name: alarm, field: detail.alarmName }- Copy the ingest URL from the source page. In EventBridge, create an API destination whose endpoint is that URL (it must be public HTTPS) and whose method is POST.
- Give the API destination a connection. Use API key authorization with header name Authorization and the value Bearer followed by an ingest-scoped API key. The key only matters when Tares has auth on.
- Create a rule on the default event bus that matches source
aws.cloudwatchand detail-typeCloudWatch Alarm State Change, and set the API destination as its target. - Flatten
detail.state.valuewith an input transformer on the rule target if you want the alarm state as a label. It is two levels deep, and a dotted label reads one level. CloudWatch alarm actions that publish to SNS do not fit, because an SNS HTTPS subscription cannot send a custom header and its confirmation message would never be confirmed. - Ask the agent to include the timing in its finding, for example: “When you write a finding, state the time of the last change before the first alarm and the gap between them.” Then check Recent events and Fields on each source page to confirm both arrive.
What you get
- One timeline per service where a merged pull request or push and the first CloudWatch alarm appear in the order they happened.
- Alarm events placed at the time CloudWatch reports, because the source reads
timeas the event time. - GitHub events that keep the same labels whether they arrive by App or by token, so a trigger written for one keeps working on the other.
- A finding that records the change, the first alarm and the gap between them, so the next investigation reads the earlier conclusion instead of starting over.
- A trigger option: a trigger on the alarm events wakes the agent, with the change already on the same timeline.
When something else fits
- If your deploys are not tagged in GitHub, such as releases cut from another system, post them to a webhook source from your CD pipeline so the change still lands on the timeline.
- If your alarms already flow through Alertmanager or Prometheus rules, use those connectors for the alert side. The EventBridge route is for CloudWatch alarms.
- If you need a commit seconds after it lands, use the GitHub App rather than a personal token, which polls every 2 minutes.
Related
Last updated on