How do I label logs from Docker containers so an AI SRE can narrow failed deploys to one service?

Tares labels Docker container logs when you declare a primary label on the docker_logs source, so one read on a service returns its logs with its metrics and alerts. Give the label a constant value or a field, mark it primary, and use the same label name on the service’s other sources. An AI SRE then reads one timeline per service, so a failed deploy points at one service instead of the whole fleet.

How it works

The docker_logs connector tails a running container’s logs and stores one event per line. Tares reads them through the Docker daemon, so run taresd on the host, or mount the Docker socket and include the Docker CLI in the image.

The label is what ties a line to a service.

• Each entry under labels has a name. Its value comes from a field in the event or from a const you write.
• The entry marked primary is the key for the source. Sources whose primary label has the same name and value share one timeline.
• When the container’s log lines, the service’s Prometheus metrics and its alerts all carry a service label, a read on that service returns them merged on one clock.

The AI SRE guide builds its demo this way. The metrics, log and alert sources are all keyed by service. The agent reads the correlated timeline for api-server and ties a failure to specific evidence lines.

Two options shape which lines land and how they are described:

• match keeps only lines that match a regex, such as ERROR or WARN. drop skips lines that match a regex, such as access logs.
• label_pattern is a regex with named groups. It turns parts of each line into per-line label context.

Set it up

1. Find the container name with docker ps. If the container runs under Compose without a fixed container_name, use the Compose service name and set compose_file to an absolute path, or a path relative to where the daemon runs, so the tail follows restarts.

2. Add the source. In the console go to Sources → Add source → docker_logs, set container, and use Discover to pick from running containers. You can also ask a connected agent to find your containers and ingest the logs. It calls discover_docker, then test_source for a dry run, then create_source. The resulting source looks like this:

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

Here service is the label name, api-server is the constant value, and primary: true makes it the key. One container usually serves one service, so a constant is enough.

3. Use the same label on the service’s other sources. Name it service on your metrics and alerts sources too, and give it the same value, such as api-server. For Prometheus metrics you can read service from the metric labels instead of a constant.

4. Trim the noise if you want. Set drop to skip access logs, or match to keep only error lines, so the lines an agent reads around a failed deploy are the ones that matter.

5. Add deploys as a source on the same key. Create a webhook source with a primary service label read from the payload’s service field and event_type: deploy. A deploy then lands on that service’s timeline next to its logs. See the AI SRE guide for the full walkthrough.

6. Verify. Open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs for that container. Then open Explore and pick the service to see logs, metrics and alerts together.

7. Read it from your agent. Start the MCP endpoint and point your client at it:

tares mcp        # MCP endpoint on http://127.0.0.1:8788/mcp

Then ask: “Use tares: what happened to api-server in the last 15 minutes?” The client calls the read tool with the service name and answers from the correlated timeline.

What you get

• Every log line from the container carries the service key, so a read on one service returns only that service’s logs, metrics, alerts and deploys.
• A failed deploy shows up as a deploy event on the timeline, with the error lines and alerts that followed it on the same clock.
• A trigger that wakes an agent hands it the correlated timeline for the affected service, so the investigation starts with the evidence already narrowed.
• The agent’s finding is written back to that service’s timeline. The next investigation of the same service starts from the previous conclusion.
• match, drop and label_pattern keep the log volume and the label context under your control.

When something else fits

• If your logs reach you through an OpenTelemetry pipeline instead of a container tail, the OTLP connector carries the service name as a resource attribute, and you label it from that path.
• If you need a container’s logs without access to the Docker daemon, docker_logs does not fit, because taresd has to be able to run docker logs for the container. Use another route into Tares for those logs.
• For the full container setup, field table and verification steps, see the Docker logs connector page.

Related