How do I use the Codex CLI to investigate Datadog monitors with Tares?

Tares lets the Codex CLI investigate Datadog monitors over MCP, reading each alert together with the logs, metrics and deploys for the same service. Datadog reaches Tares through the webhook connector and a vendor setup, and Tares has no connector of its own for Datadog today. You connect Codex to the Tares MCP endpoint, then ask about the service that alerted. Codex answers from one correlated timeline instead of from the alert text alone.

How it works

• Datadog posts to a webhook source. You add a webhook in Datadog that points at the Tares ingest URL. Tares stores each payload lossless as an event. The webhook connector maps payload fields into the event through the labels you declare, and one label is marked primary.
• A shared label ties the alert to its evidence. Give the Datadog source a service label and use the same label on your metric, log and deploy sources. Tares merges every source that carries service: checkout-api into one time-ordered timeline.
• Codex reads that timeline over MCP. The Tares MCP server exposes the read tool, which returns a correlated timeline across all sources matching a label selector. It also exposes catalog_list and catalog_describe for finding out what is ingested. Codex calls read for the service that alerted and gets the Datadog alert next to the evidence around it. See Connecting agents (MCP) for the full tool list.
• Keep the payload flat. event_type_field and text_template read top-level payload fields only, so write the Datadog payload as a flat JSON object.

The pull side works the same way in the wider AI SRE setup, where triggers can also wake an agent when an alert lands.

Set it up

1. Create a webhook source for the monitors. In the console go to Sources → Add source → webhook, or add this to your catalog. Use the label names that your other sources use, with one marked primary:

   - name: datadog-monitors
     connector: webhook
     config:
       labels:
         - { name: service, field: service, primary: true }

   Copy the ingest endpoint from the source page.

2. In Datadog, open the Webhooks integration and add a webhook whose URL is the Tares ingest URL. Write the payload in the Payload field as flat JSON built from Datadog variables such as $ALERT_ID, $EVENT_MSG and $LINK. Include a field that carries the service name, so the service label has something to read.

3. If your Tares instance has auth on, add custom headers to the webhook as JSON with an Authorization Bearer header holding an ingest-scoped API key. On an open local instance you can skip this.

4. Put @webhook- followed by your webhook’s name in the monitor message, so the monitor calls the webhook when it fires.

5. If Tares runs on your machine, start the MCP server as a second process. On a server or Tares Cloud instance it is already running.

   tares mcp --transport streamable-http --port 8788 --taresd http://localhost:8787

6. Connect Codex to the endpoint. Keep --url before the URL, because a bare URL registers a stdio server:

   codex mcp add tares --url http://localhost:8788/mcp

   For a remote, secured instance, Codex reads the credential from an environment variable. Use a key with the read scope:

   export TARES_API_KEY=<api-key>
   codex mcp add tares --url https://<your-host>/mcp \
     --bearer-token-env-var TARES_API_KEY

7. Verify the connection. Ask Codex:

   Use tares: what are you ingesting right now?

   Codex should call catalog_list and list your sources, including the Datadog one. In the console, Recent events on the webhook source shows each monitor delivery as it arrives, and Fields shows which payload fields come through.

8. When a monitor fires, ask about the service that alerted:

   Use tares: the checkout-api monitor just fired. What happened to checkout-api in the last 15 minutes?

   Codex calls read with the service label and answers from the merged timeline.

What you get

• The alert in context. Codex sees the Datadog monitor event beside the logs, metrics and deploys for the same service, ordered on one clock, so it can tie the alert to specific evidence.
• One question per incident. You name the service that alerted. You do not hand Codex a separate tool or query for each system.
• Lossless alert data. The full Datadog payload is stored as sent, so fields you did not map to labels are still there for later reads.
• Findings that carry forward. Tares agents and outside agents record findings on an entity. A later question about the same service starts from that earlier conclusion.
• A reusable setup. The same MCP connection also works for other Tares sources, so Codex can investigate anything you ingest, not only Datadog monitors.

When something else fits

• If you want an agent woken the moment a monitor fires, instead of asking Codex yourself, add a trigger and a Tares agent on top of the same webhook source. The AI SRE guide shows the pattern.
• If you use Claude Code, Cursor or another MCP client, the connection is the same endpoint and the same read tool. Only the client command differs, as listed on the agents page.
• If an alert has no matching service label on your other sources, read cannot correlate it, so align the label names first.

Related