How do I label OpenTelemetry log records so an AI SRE can narrow a cron job that stopped to one service?

Tares lets you declare a service label on your OpenTelemetry source, read from a field or set as a constant, and mark it primary. Log lines then share a key with the service’s metrics and alerts, so one read on that service returns them together. When a cron job stops, you or an agent can start from that one service instead of searching every log.

How it works

The otlp source receives OTLP/HTTP logs, traces and metrics. One source ingests every service, and a label tells Tares where in each event to find the value that identifies the service.

• A label reads its value by path. Use resourceAttributes.<key> for resource attributes, such as resourceAttributes.service.name. Use attributes.<key> for the attributes of a log record, span or data point. Use text for the rendered log line.
• There is no bare service field. A label with field: service gets no value from an OTLP event, so always write the full path.
• Mark one label primary: true. That label is the key the source is correlated by.
• A label can also take a fixed value with const instead of reading a field. Use it when the records carry no service name and the source only receives one service’s records.

Your metrics and alerts sources use the same label name, service. Because every source carries the same key, Tares builds one timeline per service. A read on that service returns metrics, logs and alerts merged on one clock. The AI SRE guide builds this wiring end to end, with all three sources keyed by service.

For a cron job that stopped, this means its log lines sit on the same timeline as the alerts and metrics for the service it belongs to. You can see where its lines end next to whatever fired around that time.

Set it up

1. Make sure the cron job’s OpenTelemetry SDK or Collector sets service.name as a resource attribute. That is the field the label reads.
2. Create the source in the console under Sources, Add source, otlp, or ask a connected agent: “Use tares to create an OTLP source keyed by service.” You can also POST /api/sources with the same object. The config needs only labels:

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: resourceAttributes.service.name, primary: true }

3. If the cron job’s records carry no service name, give them a fixed one with a constant label instead. Use this only on a source that receives just that service’s records:

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, const: api-server, primary: true }

4. Keep the label name service on your metrics and alerts sources too, for example on a prometheus or prometheus_alerts source. The shared name is what puts all three signals on one timeline.
5. Point the exporter at the daemon:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787   # OTLP/HTTP → /v1/{logs,traces,metrics}
# or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

If Tares has auth on, the exporter sends an Authorization header with the Bearer scheme holding an ingest-scoped API key. Without auth, no header is needed. For OTLP/gRPC, install the otlp-grpc extra. The receiver listens on port 4317. 6. Verify. Recent events shows logs as they arrive, and Explore lists your services under the service key. Open the service and check that its logs, metrics and alerts appear together. 7. Ask your connected MCP client about the service. The client calls read with the service name and answers from the correlated timeline:

Use tares: what happened to api-server in the last 15 minutes?

For the full connector reference, see OpenTelemetry: otlp.

What you get

• One service key shared by logs, metrics and alerts, so a single read returns all three.
• A narrower search. Instead of scanning every log line, you or an agent open one service and see its evidence on one clock.
• A setup that covers every service. The one otlp source labels each service from its resource attributes, with no per-service setup.
• The same read works interactively through your MCP client and for a trigger that wakes an agent. Both start from the correlated timeline for that service.
• A check in the console. Explore shows whether the service appears under the service key and whether its log lines landed.

When something else fits

• If the cron job runs in a container and you do not use OpenTelemetry, the docker_logs connector tails its logs and takes a constant service label with the same key.
• If you only need a stray custom event, such as a job reporting its own completion, a webhook source with a primary service label takes a JSON post from the job itself.
• If the records never carry a service name and several services share one source, a constant label would mislabel them. Set service.name at the exporter instead.

Related