Skip to Content

How do I label OpenTelemetry log records by service so they join the right timeline?

Tares labels OpenTelemetry log records by service when you declare a label named service on your otlp source, read it from resourceAttributes.service.name and mark it primary. Logs, traces and metrics from every exporter then carry that key. Any source that uses the same primary service label lands on the same timeline.

How it works

One otlp source ingests every service. Tares receives OTLP/HTTP at POST /v1/{logs,traces,metrics}, and a label reads its value from the event by path:

  • resourceAttributes.<key> reads a resource attribute, for example resourceAttributes.service.name.
  • attributes.<key> reads an attribute of a log record, span or data point.
  • text reads the rendered log line.

There is no bare service field on an OTLP event. A label written as field: service gets no value, so the log lines would not join any timeline. Use the full path resourceAttributes.service.name instead.

The primary: true flag marks the label that keys the entity. A label can also take a fixed value with const, which suits a source that belongs to one service, such as a docker_logs source. An otlp source carries many services, so read service from the resource attribute rather than a constant.

The join happens on the label name and value. Your metrics source, your alerts source and your OTLP logs each declare a label named service, marked primary on each source. Tares then merges them into one timeline per service. The AI SRE guide builds exactly this: sources keyed by the same service label, so metrics, logs and alerts share one clock when you open the service in Explore.

Set it up

  1. Start Tares and open the console. Locally that is tares up.
tares up
  1. Go to Sources, choose Add source and pick otlp. There is no required config, only labels. You can also ask a connected agent: “Use tares to create an OTLP source keyed by service.” The agent calls create_source.

  2. Declare the service label from the resource attribute and mark it primary. The same object works with POST /api/sources.

- name: otel connector: otlp config: labels: - { name: service, field: resourceAttributes.service.name, primary: true }
  1. Give the other sources for the same services a label with the same name, service, marked primary. A Prometheus metrics source reads it from a metric label, as in { name: service, field: metric.service, primary: true }. A single-container log source can use a constant, as in the docker_logs example:
- name: logs connector: docker_logs poll: 5s config: container: tares-demo-api-server # the container name (docker ps) labels: - { name: service, const: api-server, primary: true }
  1. Point your exporter at Tares. Use OTLP/HTTP, or gRPC if you installed the otlp-grpc extra.
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787 # OTLP/HTTP → /v1/{logs,traces,metrics} # or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
  1. If Tares has auth on, the exporter sends a Bearer Authorization header holding an ingest-scoped API key. Without auth on, no header is needed.

  2. Verify. Recent events shows log records as they arrive, and Explore lists your services under the service key. Pick a service to see its logs next to its metrics and alerts. The full connector reference is on the OpenTelemetry connector page.

What you get

  • Every log record from an instrumented app or an OpenTelemetry Collector is keyed by its service.name, with no per-service setup on the Tares side.
  • Log lines, metrics and alerts for the same service appear on one timeline, merged on one clock.
  • Triggers and agents that watch sources keyed by service read the same correlated evidence, so an agent investigating a service sees its OTLP logs together with the rest.
  • New services show up under the service key in Explore as soon as they export, because one source covers all of them.

When something else fits

  • If a source belongs to a single service and has no resource attributes to read, a const label gives it the fixed service value, as the docker_logs example does.
  • If your logs live in a container and you do not run OpenTelemetry, a docker_logs source is the simpler path.
  • If a label stays empty, check the field path first: resourceAttributes.service.name works, a bare service does not.

Still stuck? Get help

Last updated on