How does an AI SRE read logs from Docker containers while investigating error spikes?

Tares tails a container’s logs with the docker_logs connector, labels each line by service, and gives the agent those lines on one timeline with the alert. When an error spike fires an alert, the agent reads the error log lines and the alert on the same clock for that service. You add one source, give it a service label, and the agent can read it right away.

How it works

The docker_logs connector tails a running container’s logs and stores one event per line. It reads all lines by default, and optional match and drop regex filters narrow that down. The source polls every few seconds, so new lines show up quickly.

The label is what ties the logs to the alert. When your log source, your metrics source and your alert source all carry the same service label, Tares merges them into one timeline per service, with metrics, logs and alerts on one clock. The AI SRE guide builds this end to end.

During an error spike the flow looks like this:

• Your alerting fires, for example a high error rate alert, and the alert event lands on the service’s timeline.
• The error log lines from the container are already on that same timeline, next to the alert.
• A trigger watching the sources, keyed by service, wakes the agent. The agent reads the correlated timeline and writes a finding that ties the failure to specific evidence lines.
• You can also ask a connected agent on demand. It calls read for the service and answers from the same correlated view.

Prerequisite: Tares needs access to the Docker daemon. Run taresd on the host, or mount the Docker socket and include the Docker CLI in the image.

Set it up

1. Install Tares and start it. Open the console at http://127.0.0.1:8787.

uv tool install tares      # or: brew install glassflow/tap/tares
tares up

2. Add the log source. You have two ways:
   • Ask a connected agent: “Use tares to find my running containers and ingest the api-server’s logs.” The agent calls discover_docker to scan running containers, runs test_source as a dry run that tails a few lines, then calls create_source.
   • Or go to Sources → Add source → docker_logs and set container to the container or compose service name. Discover lists the running containers to pick from.
3. Label the source by service. This is the key step for investigations. Give the source a service label and mark it primary, as in this example:

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

4. If the container runs under Compose without a fixed container_name, set container to the service name and add compose_file, an absolute path or one relative to where the daemon runs, so the tail follows restarts.
5. Keep only what helps an investigation. Set match to keep lines that match a regex, such as ERROR|WARN. Set drop to skip lines, such as access logs. Use label_pattern, a regex with named groups, to add per-line label context.
6. Use the same service label on your metrics and alert sources, so all three land on one timeline. Prometheus metrics and Prometheus alerts both work this way.
7. Verify. Open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs for the container.
8. To investigate on demand, start the MCP endpoint in a second shell and connect your client.

tares mcp

Then ask: “Use tares: what happened to api-server in the last 15 minutes?” The client calls read for that service and answers from the correlated timeline.

What you get

• The container’s log lines on the same per-service timeline as the metrics and the alert that fired, so the agent sees which errors came with the spike.
• A trigger that wakes an agent with that timeline already assembled, so the investigation starts with the evidence in hand and no per-system lookups.
• Findings the agent writes back to the service’s timeline. The next investigation, or your connected client, reads the evidence and the earlier conclusion together.
• Less noise, because match and drop let you keep error and warning lines and skip access logs.
• One setup that works the same for a demo container and your own: you change the container name and the label.

The Docker logs connector page has the full field reference.

When something else fits

• If your services already ship logs through an OpenTelemetry pipeline, use the OTLP connector instead of tailing containers. The AI SRE guide lists both as log options.
• If your containers are not on a host where taresd can run docker logs, the connector cannot read them, so pick a connector that fits where those logs live.
• Keep alerting in the system that already owns it. Tares ingests what fired and correlates it with the logs, and it does not re-threshold anything.

Related