Skip to Content

How does an AI SRE read OpenTelemetry log records while investigating slow database queries?

Tares receives OpenTelemetry log records through one otlp source labeled by service, so an AI SRE reads your slow-query log lines on the same per-service timeline as the alert. You connect the source, point your exporter at Tares, and the agent reads the correlated timeline for the service in question. The setup needs no per-service configuration.

How it works

  • The otlp connector receives OTLP/HTTP logs, traces and metrics at /v1/logs, /v1/traces and /v1/metrics. OTLP/gRPC is available too, through the otlp-grpc extra, and listens on port 4317.
  • One source ingests every service. Resource attributes such as service.name become labels, so everything your SDK or OpenTelemetry Collector exports correlates by service.
  • A label reads its value by path. Use resourceAttributes.<key> for resource attributes, attributes.<key> for the attributes of a log record, span or data point, and text for the rendered log line. There is no bare service field, so a label with field: service gets no value from an OTLP event.
  • Tares puts logs, metrics and alerts that carry the same service label on one timeline, merged on one clock. A slow-query log line sits next to the alert that fired for that service.
  • An agent reads that timeline in two ways. A trigger can wake a Tares agent when an alert event arrives. Or you connect an MCP client and ask about a service, and the client calls read for that service. Findings the agent writes go back onto the same timeline, so the next investigation starts from the last conclusion. The AI SRE guide builds this loop end to end.

Set it up

  1. Start Tares and open the console at http://127.0.0.1:8787.

    tares up
  2. Create the OTLP source. Ask a connected agent: “Use tares to create an OTLP source keyed by service.” Or go to Sources, Add source, otlp in the console. No connector-specific config is required, only the service label. The OpenTelemetry connector page has the full reference.

    - name: otel connector: otlp config: labels: - { name: service, field: resourceAttributes.service.name, primary: true }
  3. Point your exporter at Tares. The service that runs the slow queries must set service.name as a resource attribute, because that is what the label reads.

    OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787 # OTLP/HTTP → /v1/{logs,traces,metrics} # or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
  4. If your Tares has auth on, the sender uses a Bearer Authorization header holding an ingest-scoped API key. Without auth on, the header is not needed.

  5. Verify. Recent events shows logs as they arrive, and Explore lists your services under the service key. Open the service that runs the slow queries and check that its log lines appear.

  6. Make sure the alert for the slow queries carries the same service label, so the alert and the logs land on the same timeline. Then start the MCP endpoint in a second shell and connect your client.

    tares mcp
  7. Ask the client about the service and the time of the alert, for example: “Use tares: what happened to checkout in the last 15 minutes?” The client reads the correlated timeline, including the log lines around the alert, and answers from that evidence.

What you get

  • Log lines from your OpenTelemetry pipeline on the same per-service timeline as your alerts, so the agent does not query each system separately.
  • One source for every service. A new service shows up under the service key as soon as it exports with a service.name.
  • Log text, resource attributes and record attributes all stay on the event, so slow-query details that your instrumentation puts in the log line or its attributes are there for the agent to read.
  • Investigations that build on each other. The agent’s finding is written back to the timeline, and the next read, by you or by an agent, starts with it.
  • The same evidence for interactive questions through an MCP client and for automatic first looks when a trigger fires.

When something else fits

  • If your logs live in a running container rather than an OpenTelemetry pipeline, the docker_logs connector tails them instead. The AI SRE guide compares the options for logs.
  • If you only need to investigate by hand and have no instrumentation yet, start with the guide’s demo, which wires logs, metrics and alerts for you.

Still stuck? Get help

Last updated on