How does an AI SRE read logs from Docker containers while investigating crash loops?

Tares tails a container’s logs with the docker_logs connector and labels them by service, so an agent reads log lines in the same window as the crash loop alert. Each log line becomes one event on that service’s timeline, next to its metrics and alerts. A trigger can wake an agent when the alert fires, or you can ask a connected agent to read the timeline on demand. For a Compose service, set compose_file so the tail follows container restarts.

How it works

The docker_logs connector tails a running container’s logs and stores one event per line. It keeps all lines by default, and you can narrow them with match and drop regex filters.

The connector needs access to the Docker daemon. Run taresd on the host, or mount the Docker socket and include the Docker CLI in the image.

The label is what puts logs and alerts in one window:

• You give the log source a primary service label, for example the constant api-server.
• Your metric and alert sources carry the same service label.
• Tares merges everything with that label into one timeline per service, on one clock.

When a container crash loops, the alert for that service and the log lines it printed before each restart sit side by side. An agent reads that single timeline instead of querying the alert system and Docker separately.

There are two ways an agent gets there, both described in Build an AI SRE:

• Push: a trigger watches your sources keyed by service and fires when an alert event arrives. It wakes an agent, which reads the correlated timeline and writes a finding back onto it.
• Pull: a connected MCP client asks about the service, for example “what happened to api-server in the last 15 minutes?”. The client calls read with the service name and answers from the same correlated read.

If the container runs under Compose, compose_file makes the tail follow restarts. That matters in a crash loop, because the container is replaced again and again.

Set it up

1. Start Tares and make sure it can reach Docker. Run it on the host so taresd has access to the daemon.

   tares up

2. Ask a connected agent to do the discovery for you:

   Use tares to find my running containers and ingest the api-server’s logs.

   The agent calls discover_docker to scan running containers and propose one source per candidate. It then dry-runs the source with test_source, which tails a few lines, and calls create_source.

3. Or add the source by hand. Go to Sources → Add source → docker_logs, set container to the container or compose service name, and use Discover to pick from running containers. Save with this shape:

   - name: logs
     connector: docker_logs
     poll: 5s
     config:
       container: tares-demo-api-server   # the container name (docker ps)
       labels:
         - { name: service, const: api-server, primary: true }

4. Label by service. Use the same service label name on your metric and alert sources, so all of them land on one timeline. If the label value varies per line, label_pattern takes a regex with named groups and turns them into per-line label context.

5. For a Compose container without a fixed container_name, set container to the service name and add compose_file, an absolute path or one relative to where the daemon runs. This lets the tail follow restarts.

6. Trim the noise if you need to. match keeps only lines matching a regex, such as ERROR|WARN. drop skips lines matching a regex, such as HTTP/1.1" to remove access logs.

7. Verify. Open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs on that container, since that is the access the connector uses.

8. To investigate interactively, start the MCP endpoint in a second shell and point your client at it. Then ask about the service.

   tares mcp

The Docker logs connector page has the full field table.

What you get

• Every log line from the container as its own event, labeled by service.
• One timeline per service in Explore, where the crash loop alert and the lines around it appear on the same clock.
• An agent that starts from evidence. A trigger-woken agent reads the correlated timeline and ties its conclusion to specific log lines. A connected MCP client gets the same read on demand.
• A finding written back onto the service’s timeline, so the next investigation of that service starts from the previous conclusion instead of from scratch.
• Control over volume, through match and drop, so repeated startup errors stay readable.

When something else fits

• If your containers already ship logs through an OpenTelemetry pipeline, the OTLP connector fits better than tailing each container. Build an AI SRE lists it as the alternative for logs.
• If taresd cannot reach the Docker daemon, for example because it runs somewhere with no socket access, use a connector that does not depend on Docker access.

Related