Does Tares need a public URL to receive Alertmanager alert groups?

Tares does not need a public URL for Alertmanager, only an ingest URL that Alertmanager can reach from where it runs. Alertmanager pushes alert groups to that URL, so the address can be localhost, a private network address or a public HTTPS domain. Only the network path from Alertmanager to Tares matters.

How it works

The alertmanager connector is a push receiver. Alertmanager opens the connection and POSTs a batch of alerts to a Tares ingest URL. Tares does not call out to Alertmanager.

• Direction: the connection runs from Alertmanager to Tares. Whatever address you put in webhook_configs.url must be reachable from the Alertmanager process.
• Default address: tares up serves the console and ingest on 127.0.0.1:8787. That works when Alertmanager runs on the same machine.
• One event per alert: Tares fans out the alerts array of each batch into one event per alert. event_type is the alert name, text is the summary, and the alert labels become Tares labels plus a status of firing or resolved. The full alert, including fingerprint, stays in the payload.
• Pull for contrast: a polled source such as prometheus_alerts works the other way. Tares reaches out to Prometheus on an interval (every 30 seconds by default), so Tares needs outbound access to Prometheus and nothing needs to reach Tares.

Choose the Alertmanager connector when you want Alertmanager’s grouping, silencing and resolved semantics and real-time delivery.

Set it up

1. Start Tares. The console is at http://127.0.0.1:8787.

   tares up

2. Create the source in the console under Sources, Add source, Alertmanager alerts, or ask a connected agent to create an Alertmanager source and give you the webhook URL. The source page shows a receiver block with your ingest URL filled in.

3. Add the receiver to your alertmanager.yml and reload Alertmanager:

   receivers:
     - name: tares
       webhook_configs:
         - url: http://localhost:8787/ingest/<ingest_key>
           send_resolved: true
   route:
     receiver: tares

4. If Alertmanager runs on another host, give it an address it can reach. Tares binds to 127.0.0.1 by default, so set TARES_HOST to 0.0.0.0 to expose it, then use that host’s address in the URL. For a server, the compose file with Caddy in Deployment terminates TLS on one hostname. Set TARES_DOMAIN and the ingest URL starts with your HTTPS domain, with the same /ingest/ path.

5. If Tares has auth on, add an ingest-scoped API key to the receiver as a Bearer Authorization header in the webhook_configs entry. Without auth, the receiver needs no credential. The Deployment page covers API keys and scopes.

6. Send a test batch and check Recent events:

   curl -X POST http://localhost:8787/ingest/<ingest_key> \
     -H 'Content-Type: application/json' \
     -d '{"alerts":[{"status":"firing","labels":{"alertname":"PostgresDown","severity":"critical","namespace":"prod"},"annotations":{"summary":"Postgres is down"},"startsAt":"2024-01-01T00:00:00Z"}]}'

The Alertmanager connector page has the full field reference, including key_label for choosing which alert label keys the entity.

What you get

• Alertmanager alerts arrive in real time as individual events, firing and resolved, with the full alert kept in the payload.
• Setup fits your network. Run everything on one machine with no public exposure, or put Tares on a private address or behind a TLS hostname when Alertmanager lives elsewhere.
• Tares records what Alertmanager sends and correlates it with your other sources, so you work from one timeline. The AI SRE guide shows how that feeds investigation.
• Fields on the source shows the alert labels, so you can promote namespace or service to correlate on.

When something else fits

• If Alertmanager cannot reach any address where Tares listens, a polled source such as prometheus_alerts needs no inbound connection, but you give up Alertmanager’s grouping, silencing and push delivery.
• If the sender is not Alertmanager, such as a CI system or your own service, use the webhook connector, which accepts arbitrary JSON at an ingest URL.

Related