How do I let an AI SRE look at JSON API endpoints when investigating high latency?

Tares lets an AI SRE read any JSON API endpoint by polling it with the http_poll connector, one event per item, so business data sits next to your telemetry. Add an http_poll source with the endpoint URL and a poll interval, and Tares calls the API itself. If the data lives in a database instead, the postgres connector reads new and changed rows on a cursor column.

How it works

The http_poll connector calls a JSON endpoint on the source’s schedule and stores each item as an event. You need no script on your side. This gives the AI SRE a record of the business side, such as orders or requests, in the same timeline as your metrics, logs and traces. See AI SRE for the wider picture.

How the reply becomes events:

• payload_key points at the part of the reply that holds the payload, such as data.items. Empty means the entire JSON. An object is one event and a list is one event per entry.
• id_field is the deduplication key. An item whose value was seen in recent polls is not stored again. Use the entry’s id for a list API.
• event_time_field names the field that holds the event’s time. Empty means the poll time.
• event_type is the word trigger filters match and triggers count by. event_type_field reads it from a field of the item instead.
• text_template is the one-line summary of each event, with values from the payload in braces.

The payload is stored as received, so labels you declare later can still reach every field. Nested values are reachable by dotted name in labels, trigger filters and trigger conditions.

For data in a table, the Postgres connector polls incrementally. A cursor column that only ever grows, such as an autoincrement id or updated_at, tracks progress, and each new or changed row becomes one event. A primary label on an entity column such as tenant_id makes that column the key.

Set it up

1. Pick the endpoint that holds the data you want next to your latency telemetry, such as a list of recent orders or requests.
2. Open Sources → Add source → HTTP API (poll). Fill in the method, the URL, credentials if the API wants any, extra headers, and a body for POST.
3. Press Discover. Tares calls the API once, shows the reply it got, and fills in what becomes an event. Pick the entity key from the fields it saw, then Create. You can also ask a connected agent to set it up, or send the same object to POST /api/sources.
4. Check the config. This example reads a list under data.items, deduplicates on id, and labels each event with a primary customer label, a status label and a number label for amount:

- name: recent_orders
  connector: http_poll
  poll: 1m
  config:
    url: https://api.example.com/v1/orders
    payload_key: data.items
    event_type: order
    id_field: id
    text_template: '{customer} {status} {amount}'
    labels:
      - { name: customer, field: customer, primary: true }
      - { name: status, field: status }
      - { name: amount, field: amount, type: number }

5. If the API needs a credential, put the whole value in credential and name the header in credential_header, which defaults to Authorization. The value is stored as a secret and is never returned by the API or exported:

config:
  url: https://api.example.com/v1/readings
  credential: abc123                   # an API key
  credential_header: X-API-Key

6. Open the source and check Recent events, which shows each poll’s events with the summary line. Fields lists every field seen, dotted names included, with its coverage.

To read a table instead, add a Postgres source. Enter the connection URL, press Discover, pick the table, and Tares proposes the cursor, entity key and labels from the columns:

- name: orders
  connector: postgres
  poll: 10s
  config:
    dsn: postgresql://user:pass@host:5432/appdb
    table: orders
    cursor_column: updated_at
    columns: id,tenant_id,status,amount,updated_at   # empty = every column
    labels:
      - { name: tenant_id, field: tenant_id, primary: true }
      - { name: status, field: status }

The dsn is stored as a secret. Full field lists are on the HTTP API (poll) and Postgres pages.

What you get

• Business data in the same timeline as your telemetry, so the AI SRE can see which customers, tenants or items were involved when latency rises.
• One event per item, with a summary line you control through text_template.
• Labels and number labels that triggers can watch. A trigger on a number label, such as max(windspeed) > 62 over 15m in the weather example, fires from the next reading that crosses it.
• Polling that protects the API. Tares refuses a poll under 10 seconds, and it backs off on a 429 or any 5xx, with the wait doubling from the poll interval up to an hour. A Retry-After value on a 429 is used. The source’s health shows when the next try is.
• Clear errors. Any other 4xx appears as a config error on the source’s health, with the status and a slice of the reply.
• Lossless storage, so labels you declare later still reach every field.

When something else fits

• Use the Postgres connector when the data is in a table and you want new and changed rows read on a cursor column, not a call to an API.
• Use the webhook connector when the other system should call Tares. http_poll is for APIs that Tares calls.

Related