How can an AI SRE match a jump in OpenTelemetry metrics to Docker container logs?

Tares puts OpenTelemetry metrics and Docker container logs on one timeline per service, so an AI SRE reads both on one clock and sees when a metric moved against the log lines. You create an otlp source and a docker_logs source and give both the same service label. The agent then asks Tares for that service over a window and reads metric points and log lines together.

How it works

Tares correlates sources by label. When two sources carry the same primary service label, their events merge into one timeline for that service. This is the model the AI SRE guide builds on.

• The OpenTelemetry source receives OTLP/HTTP logs, traces and metrics. Resource attributes such as service.name become labels, so the service label comes straight from your instrumentation. One source ingests every service.
• The Docker logs source tails one running container, one event per line. You set the service label yourself, as a constant, so it must match the service.name your metrics report.
• An agent connected over MCP asks for the service and a time range. For example: what happened to api-server in the last 15 minutes? It calls read for that service and answers from the merged result.

Because metric points and log lines sit on one clock, the agent can line up the moment a series jumps with the lines that appear or change around that moment. You can open the same view yourself in the console: Explore, pick the service.

Set it up

1. Start Tares and open the console at http://127.0.0.1:8787.

tares up

2. Add an OTLP source keyed by service. There is no required config, only the label.

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: service, primary: true }

3. Point your exporter, an instrumented app or an OpenTelemetry Collector, at the daemon.

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787   # OTLP/HTTP → /v1/{logs,traces,metrics}
# or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

The sender uses a Bearer Authorization header holding an ingest-scoped API key, needed only when Tares has auth on.

4. Add the container’s logs with the same service value your metrics use. Set container to the name from docker ps. If the container runs under Compose, set compose_file so the tail follows restarts.

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

taresd needs access to the Docker daemon, so run it on the host, or mount the Docker socket and include the Docker CLI in the image. Use the optional match and drop regex filters to keep only the lines you care about, or to skip access logs.

5. Start the MCP endpoint in a second shell and connect your agent. See per-client setup if you need it.

tares mcp

6. Verify. On each source, Recent events shows lines and metric points as they arrive, and Explore lists the service under the service key. Then ask your agent which service moved and what the logs said at the time.

What you get

• One timeline for the service with metric points and container log lines on the same clock.
• An agent that can say when the metric moved and which log lines appeared around that time, instead of querying a metrics tool and a log tool separately.
• Per-line filtering on the Docker side, so the timeline holds the lines that matter.
• The same evidence for you and your agent: the console Explore page and the MCP read show the same merged view.
• A path to automation. The same service key drives triggers and agents, as the AI SRE guide shows.

When something else fits

• If your containers already export logs over OpenTelemetry, the otlp source takes those logs too, so you do not need a separate Docker source.
• If your metrics live in a Prometheus server rather than an OTel pipeline, use the prometheus connector with the same service key.
• If the daemon cannot reach the Docker socket, docker_logs cannot tail the container. Fix that access first, or ship the logs another way.

Related