Skip to Content

What should I look for in OpenTelemetry log records when the problem is slow database queries?

Tares puts the log line and the record attributes of your OpenTelemetry logs on one timeline per service, so you look there for query-related text and timing values. The otlp connector receives the logs and labels them by service. An agent then reads that service’s logs next to its metrics and alerts and writes a finding tied to evidence lines. Build an AI SRE walks through the full loop.

How it works

The otlp connector receives OTLP/HTTP logs, traces and metrics at POST /v1/logs, /v1/traces and /v1/metrics. One source ingests every service. Resource attributes such as service.name become labels, so everything your SDK or Collector exports correlates by service with no per-service setup.

An OTLP log event gives you three places to look, and a label reads from one of them by path:

  • The log line. The rendered line is text. Slow-query evidence that your application writes into the message sits here.
  • Record attributes. Attributes on a log record are attributes. followed by the key. Whatever timing or query details your instrumentation attaches to a record are reachable this way.
  • Resource attributes. resourceAttributes.service.name identifies which service the slow queries belong to.

There is no bare service field. A label that uses field: service gets no value from an OTLP event.

When a trigger fires, it wakes a Tares agent, or your own agent service over a webhook. A trigger watches the sources for a service and fires on a condition you choose, built from an aggregate, a predicate and a window. The agent starts with the correlated timeline already in front of it. That timeline has the service’s logs, metrics and alerts on one clock, so the agent does not spend its first minutes querying each system. It writes a finding with what broke, since when, the likely cause tied to evidence lines and a next action. See Use AI to monitor your systems for the flow.

Set it up

  1. Start Tares locally.

    tares up
  2. Add an otlp source keyed by service. In the console go to Sources, Add source, otlp, or ask a connected agent to create an OTLP source keyed by service. The source looks like this:

    - name: otel connector: otlp config: labels: - { name: service, field: resourceAttributes.service.name, primary: true }
  3. Point your exporter at Tares.

    OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787 # OTLP/HTTP → /v1/{logs,traces,metrics} # or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

    For gRPC, install the otlp-grpc extra first. The receiver then listens on port 4317.

  4. If Tares has auth on, the exporter sends an ingest-scoped API key in an Authorization header with the Bearer scheme. Without auth, no header is needed.

  5. Check arrival. Recent events shows logs, spans and metric points as they come in. Explore lists your services under the service key. Open the service that runs the slow queries and read its log lines in context.

  6. To make timing values usable, add a label that reads the attribute your application attaches to a record, using the attributes. prefix with your own key. Set type: number on it, as you would for any numeric label. Then a trigger on the source can aggregate that value over a window.

  7. Send the other signals for the same service with the same service label, such as Prometheus metrics and alerts, so the slow queries sit next to the latency they cause. The OpenTelemetry connector page has the full reference.

What you get

  • Logs from every service in one source, correlated by service.
  • The log line and the record attributes in the same event, so you can read what your application wrote about a slow query and filter or label on the values it attached.
  • A single timeline in Explore per service, with logs, metrics and alerts on one clock. What the agent gets shows the exact text an agent reads.
  • A trigger that wakes an agent on the condition you set, and a finding written back onto the service’s timeline. The next investigation, yours or another agent’s, starts from that finding.
  • The same timeline through any MCP client, such as Claude Code or Cursor, when you want to investigate yourself. Start the endpoint with tares mcp.

When something else fits

  • If your logs come from containers rather than an OpenTelemetry pipeline, the Docker logs connector tails a running container’s logs into Tares.
  • If the slow queries show up first as a metric, such as p99 latency, the Prometheus connector brings that metric onto the same timeline.
  • If you only need to see which services are sending logs, Explore and Recent events answer that before you add any trigger.

Still stuck? Get help

Last updated on