How do I label OpenTelemetry log records so an AI SRE can narrow failed deploys to one service?

Tares lets you declare a label on an otlp source, read from a field such as resourceAttributes.service.name or set as a constant, and mark it primary. Log lines then share the service key with that service’s metrics and alerts, so one read on the service returns them together. An AI SRE asking about a failed deploy gets one correlated timeline for that service instead of every log line from every service.

How it works

One otlp source receives OTLP/HTTP logs, traces and metrics for every service. Resource attributes such as service.name become labels, so what an OpenTelemetry SDK or Collector exports correlates by service with no per-service setup.

A label reads its value from one of three places in the event, by path:

• resourceAttributes.<key> for resource attributes, for example resourceAttributes.service.name.
• attributes.<key> for the attributes of a log record, span or data point.
• text for the rendered log line.

The path is not a bare name. There is no plain service field, so a label declared with field: service gets no value from an OTLP event. Use the full path.

The label marked primary: true is the key the source is grouped by. Give your metrics, logs and alerts sources the same service label as primary, and Tares merges them onto one timeline per service. The AI SRE guide builds on exactly this: a read for one service returns metrics, logs and alerts on one clock, and a trigger wakes an agent with that same correlated view.

If a source carries logs for only one service, you can declare the label from a constant with const instead of a field, still marked primary. For a source that ingests many services, read the value from resourceAttributes.service.name so each line lands on its own service.

Set it up

1. Start Tares and open the console.

tares up

2. Go to Sources, choose Add source, then otlp. There is no required config. Add a service label read from the resource attribute, mark it primary, and save. You can also ask a connected agent to create an OTLP source keyed by service, or send the same object to POST /api/sources.

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: resourceAttributes.service.name, primary: true }

3. Make sure each app sets service.name on its OpenTelemetry resource, so the label has a value. Then point the exporter at Tares. Use the first line for OTLP/HTTP, or the second for gRPC, which needs the otlp-grpc extra.

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787   # OTLP/HTTP → /v1/{logs,traces,metrics}
# or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

4. If Tares has auth on, the exporter sends a Bearer Authorization header holding an ingest-scoped API key. Without auth, skip this.

5. Give the other signals for the same service the same primary service label. For failed deploys, add a webhook source with a primary service label read from the payload’s service field and event_type: deploy, then post one event from your CD pipeline on every deploy. The deploy then sits on the same timeline as the logs.

6. Check the result. Recent events shows logs, spans and metric points as they arrive, and Explore lists your services under the service key. Open one service and confirm its logs, metrics and alerts appear together.

7. Connect your agent to Tares so it can read per service.

tares mcp

Then ask it, for example: “Use tares: what happened to checkout in the last 15 minutes?” The client calls read for that service and answers from the correlated timeline.

What you get

• One timeline per service. Log records from the otlp source sit next to the service’s metrics and alerts, keyed by the same primary label.
• A narrow read. When a deploy fails, the agent asks about one service and gets that service’s evidence, not the logs of everything you export.
• A clear order of events. With deploy events on the same timeline, the agent can see a deploy land and the errors that follow it.
• Triggers that carry the same view. A trigger over sources keyed by service wakes an agent with the correlated timeline for the service that fired.
• No per-service setup. A new service that sets service.name shows up under the service key in Explore as soon as it exports.

When something else fits

• If a source covers a single service and has no service.name resource attribute, declare the primary label from a constant rather than a field.
• If your logs live in a container and you do not use OpenTelemetry, the docker_logs connector tails a container’s logs and takes a constant service label. The full list of connectors is on the OpenTelemetry connector page and the guide linked above.

Related