How do I label logs from Docker containers so an AI SRE can narrow crash loops to one service?

Tares lets you declare a service label on a docker_logs source and mark it primary, so every log line from that container carries the same key as the service’s metrics and alerts. A read on that service then returns the logs, metrics and alerts together on one timeline. When a container restarts repeatedly, the AI SRE sees that service’s lines and nothing from the others.

How it works

A docker_logs source tails one running container and stores one event per log line. The label you declare on the source is what ties those lines to a service.

• A label has a name and a value. The value comes from a constant (const) or from a field (field). The primary flag marks the label that keys the entity.
• For a container, the simplest value is a constant: const: api-server stamps every line from that container with service: api-server.
• Your metrics source (for example Prometheus) and your alerts source carry the same service label. Sources that share a primary label key merge into one timeline per service.
• Reading that service returns metrics, logs and alerts merged on one clock. In the AI SRE guide, the demo’s three sources are all keyed by the same service label for this reason.

For crash loops this matters because a restarting container produces a burst of startup and error lines. With the label in place, those lines sit next to the alert that fired and the metric that moved, all under one service name.

Set it up

1. Find the container name with docker ps. If the container runs under Compose, note the service name and the path to the compose file. Tares needs access to the Docker daemon, so run taresd on the host, or mount the Docker socket and include the Docker CLI in the image.

2. Add the source under Sources → Add source → docker_logs, or ask a connected agent to find your containers and ingest the logs. The agent calls discover_docker, then test_source for a dry run, then create_source. The object looks like this:

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

3. Use the same label name and value on your other sources for that service. For a Prometheus source, the label reads the service from the metric labels:

labels:
  - { name: service, field: metric.service, primary: true }

4. If the container runs under Compose without a fixed container_name, set container to the service name and add compose_file, an absolute path or one relative to where the daemon runs. This makes the tail follow restarts, which is what you want when a container keeps cycling.

5. Narrow the noise if you need to. match keeps only lines that match a regex, such as ERROR|WARN. drop skips lines that match a regex, such as HTTP/1.1" to drop access logs. label_pattern is a regex with named groups that gives per-line label context.

6. Verify. Open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs <container>.

7. Start the MCP endpoint and ask your client about the service:

tares mcp

Use tares: what happened to api-server in the last 15 minutes?

The client reads the correlated timeline for api-server. The log lines from the restarting container appear with that service’s metrics and alerts.

What you get

• Log lines from one container labeled as one service, so a read on that service returns only its evidence.
• One timeline per service that merges logs, metrics and alerts, so a restart burst in the logs lines up with the alert that fired.
• A tail that follows Compose restarts when you set compose_file.
• Filters with match and drop that keep the timeline readable during a noisy crash loop.
• The same service key for triggers, so a trigger that watches your sources by service can wake an agent with the correlated evidence.

The Docker logs connector page has the full field table.

When something else fits

• If your logs already flow through an OpenTelemetry pipeline, use the OTLP connector and label the service from the resource attributes instead of tailing containers.
• If the process runs outside Docker, a different source type fits better. The AI SRE guide lists the connector choices per signal.

Related