How do I read JSON API endpoints next to Grafana Loki logs when an AI SRE investigates?

Tares reads a JSON API with the http_poll connector and your Grafana Loki logs with the loki connector, and puts both on one timeline when their events carry the same label value. Give both sources a label such as service or tenant_id, and one read of that value returns the API items and the log lines together. The AI SRE guide shows how an investigation uses that shared timeline.

How it works

Each source turns what it reads into events, and each event carries the labels you declare on its source. A shared label value is what puts an event from one source next to an event from another.

• Loki logs. The loki connector selects log lines with a LogQL query and polls every 5 seconds by default. It also works with Grafana Cloud. Its settings are url, query, bearer_token, username, password, tenant_id, match, drop and key.
• JSON API endpoints. The http_poll connector calls an endpoint on the source’s schedule, every 5 minutes by default, and stores one event per item. Nested values are reachable by dotted name, so a label can read a field such as current_weather.windspeed. See HTTP API (poll) for the full reference.
• The shared label. Declare a label with the same name on both sources, for example service or tenant_id. Mark it primary on the source where it identifies the entity. Events from both sources that carry the same value for that label then sit on the same timeline.

The http_poll payload is stored as received, so a label you declare later can still reach every field of earlier events.

Set it up

1. Pick the shared value. Choose a field that both the log lines and the API items carry, such as service or tenant_id.
2. Add the Loki source. Create a loki source with the Loki url, your LogQL query, and credentials if the instance needs them (bearer_token, or username and password, plus tenant_id for a multi-tenant setup). Declare a label with your shared name on it. Use match and drop to keep only the lines you need.
3. Add the API source. In Sources → Add source → HTTP API (poll), enter the method, URL and any credential, then press Discover. Tares calls the API once, shows the reply, and fills in what becomes an event. Pick the entity key from the fields it saw, then create the source.
4. Declare the same label on the API source. Point it at the field that holds the shared value:

- name: service_status
  connector: http_poll
  poll: 5m
  config:
    url: https://api.example.com/v1/readings
    credential: abc123
    credential_header: X-API-Key
    payload_key: data
    id_field: id
    event_type: api_reading
    labels:
      - { name: service, field: service, primary: true }

5. Set the event fields. payload_key points at the part of the reply that holds the items, and a list becomes one event per entry. id_field is the deduplication key, so an item seen in recent polls is not stored again. event_time_field names the field holding the event’s time. If you leave it empty, the poll time is used.
6. Keep the poll interval sane. The connector refuses a poll under 10 seconds. On a 429 or a 5xx it backs off, doubling the wait up to an hour, and the source’s health shows when the next try happens.
7. Verify. Open each source. Recent events shows the events with their summary lines, and Fields lists every field seen with its coverage. Confirm that the shared label is present on both sources.

If the shared value lives in your own database rather than behind an API, the Postgres table connector reads rows the same way, and an entity column such as tenant_id becomes the key through a primary label.

What you get

• API items and log lines for the same service or tenant appear on one timeline, and reading that value returns both.
• Each poll adds a fresh API reading beside the log lines that carry the same value.
• One event per API item, with a summary line you control through text_template, so a reading is quick to scan beside a log line.
• Deduplication through id_field, so an unchanged item does not fill the timeline.
• A credential for the API that is stored as a secret, never returned by the API and never exported.
• Visible source health: rate limits and config errors show on the source itself.

The AI SRE guide describes how an investigation works from this data.

When something else fits

• If the source of truth is a Postgres table rather than an API, use the Postgres table connector. It reads new and changed rows on a cursor column.
• If a system should call Tares instead of Tares calling it, use the webhook connector.
• If the two sources share no label value, they will not line up. Add a field both sides carry, or declare a label that maps one side’s field to the other’s name first.

Related