How do I read JSON API endpoints next to OpenTelemetry log records when an AI SRE investigates?

Tares puts JSON API items and OpenTelemetry log records on one timeline when both carry the same label value, such as a service name. You create an otlp source for the logs and an http_poll source for the API, and give both a label with the same name and value. An AI SRE that reads that value then gets log lines and API items together. This is part of the AI SRE workflow.

How it works

Tares stores events from every source and groups them by label values. Two sources land on the same entity only when they carry the same label name and the same value.

• otlp receives OTLP/HTTP logs, traces and metrics. Resource attributes become labels, so the service label reads resourceAttributes.service.name. A label reads its value by path, and there is no bare service field, so field: service gets no value from an OTLP event.
• http_poll calls a JSON endpoint on a schedule and stores one event per item. An object in the reply is one event, and a list is one event per entry. A label can read a field of the item by dotted name, or hold a fixed value with const.
• The join is the label. If the log lines carry service equal to checkout, the API items need service equal to checkout too. It can be a service, a tenant_id or any other key your logs also carry.

The http_poll payload is stored as received, so labels you declare later can still reach every field. Reading the shared value returns the log records and the API items in one timeline, and the AI SRE sees what the API reported next to what the service logged.

Set it up

1. Create the otlp source. It needs no connector-specific config, only a label for the service. See OpenTelemetry for the full page.

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: resourceAttributes.service.name, primary: true }

2. Point your exporter at Tares. When Tares has auth on, the sender uses a Bearer Authorization header holding an ingest-scoped API key. Without auth on, you do not need it.

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787   # OTLP/HTTP → /v1/{logs,traces,metrics}
# or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

3. Add the http_poll source with a label that has the same name as the log label. Use const when the whole endpoint belongs to one service, or field when each item carries the value. The page for HTTP API (poll) lists every setting.

- name: weather_berlin
  connector: http_poll
  poll: 10m
  config:
    url: https://api.open-meteo.com/v1/forecast?latitude=52.52&longitude=13.41&current_weather=true
    event_type: weather
    event_time_field: current_weather.time
    id_field: current_weather.time
    text_template: 'Berlin wind {current_weather.windspeed} km/h, {current_weather.temperature} C'
    labels:
      - { name: city, const: berlin, primary: true }
      - { name: windspeed, field: current_weather.windspeed, type: number }

This example is the shipped one, with a city label. For your own endpoint, name the label service instead and give it the value your OTLP logs carry as service.name.

4. Set id_field to the item’s own id so a repeated poll does not store the same item twice, and set event_time_field so items sit at their real time on the timeline. Leave event_time_field empty and the poll time is used.

5. If the API needs a credential, set credential and credential_header. The value is stored as a secret and is never returned by the API.

6. Verify. On the otlp source, Recent events shows logs as they arrive and Explore lists services under the service key. On the http_poll source, Recent events shows each poll’s events and Fields lists every field seen. Confirm both sources show the same value under the shared key.

What you get

• One read of a label value returns the log lines and the API items together, so the AI SRE does not read two separate histories.
• API items sit at their own event time when you set event_time_field, so an API status change lines up with the log lines around it.
• Number labels, such as windspeed in the example, can be used in trigger conditions like max(windspeed) > 62 over 15m.
• Polling is handled for you. http_poll refuses an interval under 10 seconds, backs off on a 429 or any 5xx, and shows the problem in the source’s health.
• One otlp source covers every service that exports to it, so adding a service does not mean adding a source.

When something else fits

• If the data you want next to the logs is rows in your own database, use the Postgres table connector. It reads new and changed rows and makes a column such as tenant_id the key through a primary label.
• If the API can call Tares itself, the webhook connector fits better than polling.
• If the API and the logs share no value, there is nothing to join on. Add a field to one side first, or use a fixed const label on the API source that matches the service name.

Related