How do I label Docker container logs so an AI SRE can narrow slow database queries to one service?

Tares lets you label Docker container logs by declaring a service label on the docker_logs source and marking it primary. Every log line then shares a key with that service’s metrics and alerts. A read on that service returns the slow query lines together with the evidence around them, so an AI SRE looks at one service instead of every container. You set the label once, in the source config or in the console.

How it works

The docker_logs connector tails a running container’s logs and stores one event per line. Each source can declare labels. A label has a name, a value taken from a field or from a const, and an optional primary flag.

The primary label is the key Tares groups events by. If your metrics source, your alerts source and your log source all carry a primary service label with the same value, Tares builds one timeline for that service. In the AI SRE guide the three demo sources are keyed this way, so metrics, logs and alerts merge on one clock for api-server.

For a container whose logs all belong to one service, a constant is the simplest label. Every line from that container gets the same service value, and you do not need to parse the line.

Two more settings keep the log stream useful for query work:

• match keeps only lines that match a regex, for example ERROR|WARN.
• drop skips lines that match a regex, for example HTTP/1.1" to drop access logs.

The connector also has label_pattern, a regex with named groups that adds per-line label context.

To query, an agent calls read with the service name. The correlated read returns that service’s metrics, logs and alerts together, so slow query lines sit next to the latency metric and the alert that fired.

Set it up

1. Find the container name with docker ps. If the container runs under Compose without a fixed container_name, use the Compose service name and also set compose_file. Use an absolute path, or a path relative to where the daemon runs, so the tail follows restarts.
2. Make sure taresd can reach the Docker daemon. Run taresd on the host, or mount the Docker socket and include the Docker CLI in the image. Start Tares if it is not running:

tares up

3. In the console, go to Sources, then Add source, then docker_logs. Set container. Discover lists the running containers to pick from. Or ask a connected agent to find your running containers and ingest the logs. It uses discover_docker, test_source and create_source.
4. Declare the label. Use the same service value that your metrics and alerts sources use, and mark it primary:

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

5. If the service is a database or a service that logs its slow statements, narrow the stream with match so only the lines you care about arrive. Use drop to remove noise such as access logs. Both take regular expressions.
6. Save, then open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs <container>, because that is the access the connector needs.
7. Open Explore and pick the service. The log lines appear on the same timeline as the metrics and alerts for that service. Connect an MCP client with tares mcp and ask it what happened to the service in the last 15 minutes. The client calls read for that service and answers from the correlated result.

You can also create the same object with POST /api/sources. The connector’s own page, Docker logs, lists every field.

What you get

• One key per service. The primary service label ties the log lines to the metrics and alerts that carry the same label, so a read on that service returns all three.
• Narrow reads. An AI SRE asking about one service gets that service’s slow query lines and nothing from other containers.
• Evidence in context. A slow query line appears beside the latency metric and any fired alert on the same clock, so the agent can tie a symptom to specific lines.
• Less noise. match and drop filter lines before they reach the timeline, so the agent reads the lines that matter.
• A timeline that triggers can use. A trigger keyed by service watches these log events with the same key as your other sources. Findings the agent writes land on the same service timeline, so the next investigation starts from them.

When something else fits

• If your services already ship logs through an OpenTelemetry pipeline, the AI SRE guide points to the OTLP connector as the alternative to per-container tailing.
• If the database is not in a container on the host that taresd can reach, tailing with docker_logs does not apply. Pick a connector that fits where those logs live.

Related