How do I label logs from Docker containers by service so they join the right timeline?

Tares joins Docker container logs to a service’s timeline when the docker_logs source declares a label and marks it primary. Add a label from a constant or a field, for example service, and use the same label name on the service’s metrics and alert sources. Tares then merges logs, metrics and alerts for that service on one clock.

How it works

• A docker_logs source tails one running container and creates one event per log line.
• Each source has a labels list. A label has a name and takes its value from a field of the event (field) or from a fixed value (const).
• Setting primary: true marks the label that keys the source. Sources that carry the same primary label, such as service, share a key.
• Events that share that key land on one timeline per service. In the AI SRE demo, the metrics, logs and alerts sources are all keyed by the same service label, so Explore shows them merged for api-server.
• For a container’s logs, a constant is the simplest choice: the whole container belongs to one service, so you write the service name once.
• If you need context that changes per line, label_pattern takes a regex with named groups and turns them into per-line label context.

Set it up

1. Find the container name with docker ps. If the container runs under Compose without a fixed container_name, use the Compose service name instead and also set compose_file (an absolute path, or relative to where the daemon runs) so the tail follows restarts.
2. In the console, go to Sources → Add source → docker_logs. Set container, or use Discover to pick from the running containers.
3. Add a label named service with a constant value and mark it primary. Use the exact label name and value that your metrics and alert sources use for this service.
4. Save the source. This is the full object:

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

5. Optionally narrow the lines with match (keep only lines matching a regex, such as ERROR|WARN) or drop (skip lines matching a regex, such as HTTP/1.1” to drop access logs).
6. Verify. Open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs for the container, because it needs the same access. Run taresd on the host, or mount the Docker socket and include the Docker CLI in the image.
7. Open Explore and pick the service. Its logs now appear next to its metrics and alerts. The full field list is on the Docker logs connector page.

You can also ask a connected agent: “Use tares to find my running containers and ingest the api-server’s logs.” The agent scans the running containers with discover_docker, dry-runs with test_source, then calls create_source.

What you get

• Log lines keyed by service, on the same timeline as that service’s metrics and alerts.
• One correlated read per service. A trigger that watches the sources keyed by service wakes an agent with the logs next to the evidence that fired it.
• A filtered log stream, if you set match or drop, so the timeline holds the lines that matter.
• Lines you can check in Recent events on the next poll.

When something else fits

• If your services ship logs through an OpenTelemetry pipeline instead of container tails, use the OTLP connector and keep the same primary service label.
• If the service name is already inside each log event and differs per line, read the label from a field instead of a constant.

Related