How do I label logs from Docker containers so an AI SRE can narrow a disk filling up to one service?

Tares labels Docker container logs with a service label you declare on the docker_logs source and mark primary, so an AI SRE can narrow a full disk to one service. Give the log source the same service value as that service’s metrics and alerts. A read on that service then returns its logs, metrics and alerts together on one timeline. The Docker logs connector tails the container, and the label does the correlating.

How it works

The docker_logs connector tails a running container’s logs and stores one event per line. You attach labels to the source, and each label takes its value from a field or from a constant. Mark one label primary: true. That label is the key Tares uses to group events into one timeline per service.

When your metrics source and your alerts source carry the same service label, all three signals share that key. Tares merges them on one clock. This is the read an AI SRE works from, as described in Build an AI SRE.

For a disk that is filling up, this means you do not search every log in the system. You ask for one service. The agent sees that service’s log lines, the metric that moved and the alert that fired, side by side.

A few things to know about the connector:

• Every line becomes an event by default. Use match to keep only lines matching a regex, such as ERROR|WARN, or drop to skip lines, such as HTTP/1.1" to drop access logs.
• label_pattern takes a regex with named groups and turns them into per-line label context.
• compose_file makes the tail follow restarts when the container runs under Compose.
• Tares needs access to the Docker daemon. Run taresd on the host, or mount the Docker socket and include the Docker CLI in the image.

Set it up

1. Install Tares and start it.

   uv tool install tares          # or: brew install glassflow/tap/tares
   tares up

2. Find the container name with docker ps. If the container runs under Compose without a fixed container_name, use the service name and set compose_file to an absolute path, or a path relative to where the daemon runs.

3. In the console, go to Sources, Add source, docker_logs. Set container, or use Discover to pick from the running containers. Add a label that names the service and mark it primary. The same source as YAML:

   - name: logs
     connector: docker_logs
     poll: 5s
     config:
       container: tares-demo-api-server   # the container name (docker ps)
       labels:
         - { name: service, const: api-server, primary: true }

   You can also send the same object with POST /api/sources, or ask a connected agent to find your containers and create the source for you.

4. Give the service’s metrics source and alerts source a service label with the same value. For a prometheus source, you can read service from the metric labels instead of a constant. The value must match what you set in the log source, here api-server. Keep the label name service on all three sources.

5. Check the result. Open the log source in the console and look at Recent events. Lines appear on the next poll. If nothing arrives, check that taresd can run docker logs for that container, since it needs the same access.

6. Start the MCP endpoint in a second shell and point your client at it.

   tares mcp

   Then ask: what happened to api-server in the last 15 minutes? The client calls read with the service api-server and answers from the correlated timeline.

What you get

• One timeline per service. The container’s log lines, the service’s metrics and its alerts sit together, so a disk filling up points at one service instead of the whole host.
• A read scoped to that service. An AI SRE, or you through an MCP client, asks for api-server and gets the evidence for it only.
• Less noise in the logs. With match and drop, the timeline keeps the lines that matter, such as errors and warnings, and skips access logs.
• A trigger that wakes an agent with the right context. A trigger keyed by service can fire on an alert and hand the agent the correlated timeline for that service. The agent writes its finding back onto the same timeline, so the next investigation starts from it.
• A setup you can repeat. The label is ordinary source configuration, so you declare it once per container and reuse the pattern for each service you run.

When something else fits

• If your services already ship logs through an OpenTelemetry pipeline, the otlp connector fits better than tailing containers. Label the service from its resource attributes there.
• If the container is not reachable from the host where Tares runs, docker_logs cannot tail it, so pick a connector that receives logs by push.
• If you only need alerts and not log lines, prometheus_alerts or alertmanager alone gives the agent the signal, but the logs add the evidence of what the service was writing.

Related