What should I look for in OpenTelemetry log records when the problem is error spikes?

Tares receives OpenTelemetry log records through the otlp connector, so you can look for error spikes per service on one timeline. Watch the log line text, the record attributes and the service resource attribute, and compare how many error lines land per window instead of reading each line. A trigger or a connected agent then starts from the correlated evidence for that service.

How it works

The otlp connector receives OTLP/HTTP logs, traces and metrics. One source ingests every service, and resource attributes such as service.name become labels, so everything an OTel SDK or Collector exports correlates by service.

Three places in an OTLP log event matter when you hunt an error spike:

• The log line. It is available as text. This is where the telltale error lines show up, and what an agent cites as evidence.
• Record attributes. Attributes of a log record are available as attributes.<key>, so you can label or group by any attribute your application sets.
• The service. The resource attribute resourceAttributes.service.name is the natural key. There is no bare service field, so a label written as field: service gets no value from an OTLP event. If an exporter does not set service.name, the service label has no value for its records.

A spike is a change in counts, not a single line. Triggers express that: a trigger condition is an aggregate over a window, for example a count with a predicate over 5 minutes. When the condition holds, the trigger wakes a Tares agent, or posts the timeline to your own agent service over a webhook. The agent starts with the correlated timeline for that service in front of it, with logs, metrics and alerts on one clock, and writes a finding that ties the failure to specific evidence lines.

When you investigate yourself, a connected MCP client reads the same timeline. Ask it what happened to a service in the last 15 minutes and it answers from the correlated read, including any earlier finding.

Set it up

1. Start Tares and open the console at http://127.0.0.1:8787.

tares up

2. Go to Sources → Add source → otlp. No connector-specific config is required, only labels. Key the source by service so each service’s error lines sit on their own timeline.

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: resourceAttributes.service.name, primary: true }

3. Point your exporter at Tares. OTLP/HTTP goes to /v1/logs, /v1/traces and /v1/metrics. For OTLP/gRPC, install the otlp-grpc extra and use port 4317.

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787   # OTLP/HTTP → /v1/{logs,traces,metrics}
# or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

4. If Tares has auth on, the exporter sends an Authorization header with the word Bearer followed by an ingest-scoped API key. Without auth on, you do not need it.

5. Open Recent events to see log records as they arrive, and Explore to list your services under the service key. During a spike, open the affected service in Explore and read the error lines next to the metrics and alerts on the same clock.

6. To have an agent look at the spike for you, follow Build an AI SRE. It walks through a trigger that wakes an agent, an error_spike scenario you can cause on purpose, and the connected MCP client. You start the MCP endpoint with tares mcp in a second shell.

What you get

• Error lines from every service in one source, correlated by the service label wherever your exporters set service.name.
• Log text, record attributes and resource attributes available as labels, so you group a spike the way your application describes itself.
• Triggers that fire on a windowed aggregate, so the wake-up is based on how many error lines arrived, not on someone spotting a line.
• A finding on the service’s timeline that names what broke, since when, the likely cause tied to evidence lines and a next action. The next investigation, yours or another agent’s, starts from that finding.
• The same evidence through MCP clients such as Claude Code or Cursor when you want to investigate interactively.

The OpenTelemetry connector page has the full list of label paths and exporter settings.

When something else fits

• If your logs live in a running container, the Docker logs connector tails them without an OTel pipeline. The AI monitoring guide lists it next to OTLP and Vercel logs.
• If Prometheus or Alertmanager already fires on the error rate, keep alerting there. Tares ingests what fired and correlates it with the log evidence, and the trigger wakes the agent from that.

Related