What should an AI SRE read next to alerts from Prometheus for failed deploys?

Tares puts the same service’s metrics, logs and recent deploy and config changes on one timeline beside the Prometheus alert, so an AI SRE reads that timeline and cites evidence. Key every source by one service label. The agent then sees the alert, the metric that tripped it, the error logs and the deploy that landed just before. The AI SRE guide builds this end to end.

How it works

The prometheus_alerts connector polls Prometheus’s own alerts endpoint every 30 seconds by default. Prometheus keeps owning alerting, so the alert for a failed deploy is whatever your alerting rule fires. Tares does not re-threshold anything.

Each active alert becomes one event:

• event_type is the alert name.
• text is the alert summary.
• The alert labels become Tares labels, plus a state of firing, pending or resolved.
• State is tracked across polls, so an alert fires once and a resolved event lands when it clears.

The alert is the start of the investigation. Because the other sources carry the same service label, Tares merges them into one timeline per service. For a failed deploy, an AI SRE should read these next to the alert:

• Metrics from the Prometheus connector source: 5xx rate, p99 latency, connection counts, dependency health.
• Logs from docker_logs or OTLP: the error lines around the alert time.
• Deploys and config changes from a webhook source, with event_type set to deploy or config. Most real root causes are a change, so this is the highest-value source to add.
• Earlier findings that agents already wrote onto that service’s timeline, so the next investigation starts from the previous conclusion.

A trigger watching these sources can wake an agent when an alert event arrives. The agent reads the correlated timeline and writes a finding that ties the failure to specific evidence lines.

Set it up

1. Add the alerts source. Point it at your Prometheus and keep service as a label. Tares only proposes the always-present labels severity and state, so add service from the Fields tab once real alerts have been ingested.

- name: alerts
  connector: prometheus_alerts
  poll: 30s
  config:
    url: http://localhost:9090
    severities: critical,warning   # blank = all
    labels:
      - { name: severity, field: labels.severity, primary: true }
      - { name: state, field: state }

2. Keep your metrics and log sources keyed by the same service label, with the prometheus connector for metrics and docker_logs or OTLP for logs. The label is what puts them on one timeline.

3. Add a deploy source. Create a webhook source with a primary service label read from the payload and event_type set to deploy. Creating it gives you a stable ingest URL.

- name: deploys
  connector: webhook
  config:
    event_type: deploy
    text_template: 'deploy {commit} by {author}: {message}'
    labels:
      - { name: service, field: service, primary: true }

4. Add one line to your CD pipeline, run on every deploy. Each POST lands as a deploy event on that service’s timeline.

curl -X POST https://<your-tares>/ingest/<ingest_key> \
  -H 'content-type: application/json' \
  -d '{"service": "checkout", "commit": "9f2c81d", "author": "dana", "message": "raise db pool size"}'

5. Verify. Open the alerts source: Recent events shows one event per active alert, and Fields shows the labels you can promote for correlation. Then open Explore, pick the service and check that the alert, metrics, logs and deploy appear on one clock.

6. Connect an MCP client to investigate on demand. Start the endpoint with:

tares mcp

Then ask: “Use tares: what happened to checkout in the last 15 minutes?” The client reads the correlated timeline, including the deploy event and any earlier finding.

What you get

• A diagnosis that cites evidence. The agent can say “deploy landed 14:02, errors started 14:04” and point to the alert, the metric and the log lines.
• One read instead of one tool per system. The agent works from a single per-service timeline.
• An alert that fires once per state change, with a resolved event when it clears, so the timeline stays readable.
• Findings that persist. The conclusion is written back onto the service’s timeline, so each investigation leaves the next one a better start.
• The choice of who gets woken: a Tares agent, your own agent service over a webhook, or a Slack channel.

When something else fits

• If your alerts already route through Alertmanager, use the alertmanager connector instead and keep the same service label so the timeline still lines up.
• If you only need to browse the raw alert list, the alerts source’s Recent events view is enough. Add the deploy source once you want the agent to tie failures to changes.

Related