How do I let an AI SRE look at JSON API endpoints when investigating 5xx errors on an API?

Tares polls any JSON endpoint with an http_poll source, or reads Postgres rows on a cursor, so an AI SRE can see who a 5xx error hit. Tares calls the endpoint itself, with no script on your side, and stores one event per item. To put those events next to your telemetry for the same API, give both sources a label with the same name and the same value. Start from the AI SRE guide for the wider picture.

How it works

An http_poll source polls a JSON endpoint on its schedule and stores one event per item. The reply becomes events like this:

• payload_key points at the part of the reply that holds the payload, such as data.items. Empty means the entire JSON. A list gives one event per entry.
• id_field is the deduplication key, so an item seen in recent polls is not stored again. Use the entry’s id for a list API.
• event_time_field names the field that holds the event’s time. Empty means the poll time.
• event_type is the word trigger filters match and triggers count by. text_template is the one-line summary of each event.

The payload is stored as received, so labels you declare later can still reach every field, dotted names included.

The Postgres connector is the other route. It reads a table incrementally, one event per new or changed row, tracking a column that only grows, such as an autoincrement id or updated_at. A primary label such as tenant_id becomes the entity key.

How the business data meets the telemetry. Tares puts two sources on one entity only when they carry a label with the same name and the same value. Polling an endpoint does not by itself place its items next to your logs, traces or metrics. For example, an OTLP source labels its events with service taken from resourceAttributes.service.name. To connect an http_poll source to it, give that source a service label whose value is the same as the one the OTLP source carries. A Postgres source keyed on tenant_id does not meet an OTLP source keyed on service unless you add a shared label, such as a service label with a constant value, to the Postgres source.

Two behaviors matter when the API you poll is the one failing. The connector refuses a poll under 10 seconds. On a 429 or any 5xx it backs off, doubling the wait from the poll interval up to an hour, and the source’s health says when the next try is.

Set it up

1. Pick what shows who was affected: a JSON endpoint such as a failed-requests or customer-impact export, or a table such as orders.
2. Add the source. In the app, go to Sources → Add source → HTTP API (poll), fill in the method, URL, credential and headers, and press Discover. Tares calls the API once, shows the reply and fills in what becomes an event. Pick the entity key and create it. You can also ask a connected agent, or send POST /api/sources with the same object.
3. Declare the shared label. This example polls an endpoint and labels each event with the same service value the OTLP source uses. Replace the URL with your endpoint:

- name: api_readings
  connector: http_poll
  poll: 10m
  config:
    url: https://api.example.com/v1/readings
    credential: abc123                   # an API key
    credential_header: X-API-Key
    event_type: api_reading
    labels:
      - { name: service, const: checkout-api, primary: true }

4. Make the telemetry side carry the same label name and value. On the OTLP source, the service label reads the service name from the event:

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: resourceAttributes.service.name, primary: true }

The service in your OTLP data must report checkout-api as its service.name for the two sources to share an entity.

5. For table rows, add a Postgres source and a service label next to the entity key:

- name: orders
  connector: postgres
  poll: 10s
  config:
    dsn: postgresql://user:pass@host:5432/appdb
    table: orders
    cursor_column: updated_at
    columns: id,tenant_id,status,amount,updated_at   # empty = every column
    labels:
      - { name: tenant_id, field: tenant_id, primary: true }
      - { name: service, const: checkout-api }

6. Verify. Open each source: Recent events shows each poll’s events with the summary line, and Fields lists every field seen with its coverage. Use it to confirm the key and the shared label are reliably present. See the HTTP API connector page for every field.

What you get

• Business data the agent can read while it investigates: which tenants, orders or customers sit behind a burst of 5xx responses.
• Events that share an entity with your telemetry for the same service, because the sources carry the same label name and value.
• Rows and items stored as received, so you can add labels later and still reach every field.
• A source whose health tells you when the API rate limits or rejects the poll, with the status and a slice of the reply for any other 4xx.
• Triggers that can watch the data. A number label on a polled field can drive a condition such as a maximum over a window.

When something else fits

• If a system can call Tares, such as a CI or alerting tool, use the webhook connector instead of polling.
• If the data lives in a table and you want every changed row with a cursor, use the Postgres connector rather than an API in front of it.
• If you only need the 5xx telemetry itself, such as logs, traces and metrics, an OTLP source covers that, and the polled data adds the who-was-affected side.

Related