How do I keep the credentials for JSON API endpoints safe when an AI SRE looks into error spikes?

Tares keeps the credentials for a JSON API endpoint on the source, stored as a secret, so the AI SRE investigates error spikes from the events the source stores and not from the connection. The http_poll connector sends one credential in one header with each request and never returns it through the API or exports it. A Postgres source works the same way with its connection string.

How it works

Tares calls your endpoint itself on the source’s schedule and stores one event per item in the reply. The connection details stay in the source’s config, and what you investigate is the stream of events.

• One credential, one header. In the http_poll connector, credential holds the whole value, such as Bearer xyz or an API key. credential_header names the header it travels in, and Authorization is the default.
• Stored as a secret. The credential is never returned by the API and never exported.
• Other headers stay separate. Anything else the API wants on every request goes in headers, which is not secret. Keep tokens out of it.
• Events carry the data. The payload is stored as received. Labels, trigger filters and trigger conditions can reach any field by dotted name, for example current_weather.windspeed.
• Postgres follows the same pattern. The dsn is the source’s connection URL. It is stored as a secret, redacted in the API and omitted from catalog exports.

The agent that looks into an error spike works from the timeline of stored events. See AI SRE for how that investigation runs.

Set it up

1. Open Sources → Add source → HTTP API (poll). Fill in the method, the URL, and the credentials if the API wants any: one header name and a value. Add extra headers or a POST body if you need them.
2. Press Discover. Tares calls the API once, shows the reply it got and fills in what becomes an event. Pick the entity key from the fields it saw, then create the source.
3. Or declare the source yourself. A bearer token goes in credential with Authorization as the header:

config:
  url: https://api.github.com/user
  credential: Bearer ghp_xxx          # a bearer token
  credential_header: Authorization

4. For an API key in a custom header, change the header name:

config:
  url: https://api.example.com/v1/readings
  credential: abc123                   # an API key
  credential_header: X-API-Key

5. Set event_type, which trigger filters match and triggers count by, so error events are easy to pick out. Use id_field for deduplication, and payload_key when the items sit under a key such as data.items. Declare labels for the fields you want to watch, with one primary.
6. If the errors live in a database, add a Postgres source. Enter the Connection URL (dsn) and press Discover. Tares lists the tables it can see, and you pick one and choose which columns to pull:

- name: orders
  connector: postgres
  poll: 10s
  config:
    dsn: postgresql://user:pass@host:5432/appdb
    table: orders
    cursor_column: updated_at
    columns: id,tenant_id,status,amount,updated_at   # empty = every column
    labels:
      - { name: tenant_id, field: tenant_id, primary: true }
      - { name: status, field: status }

7. Add a trigger that counts error events over a window, or compares a number label over a window, such as max(windspeed) > 62 over 15m. It fires from the next reading that crosses the threshold.
8. Open the source and check Recent events for each poll’s events. Fields lists every field seen with its coverage.

You can also ask a connected agent to create the source from a goal. For the generic polling path, discover_source reads one reply and fills in the rest. Both sources can also be created with POST /api/sources and the same object. Details are on the HTTP API (poll) page.

What you get

• Secrets stay on the source. The credential and the dsn are stored as secrets, hidden from API responses and left out of exports. The agent investigating an error spike works from events and does not need the connection.
• Request pressure is limited. The connector refuses a poll under 10 seconds when the source is saved. On a 429 or any 5xx it backs off, doubling the wait from the poll interval up to an hour, and it honors Retry-After on a 429. The source’s health shows the next try time. Any other 4xx appears as a config error on the source’s health, with the status and a slice of the reply, so a bad or expired credential is visible. The next good poll clears the backoff.
• Complete event data. Because the payload is stored as received, labels you declare later can still reach every field. You do not need to widen the credential’s scope or re-poll to add one.
• A clear check. Recent events and Fields show what came in, so you can confirm the error data arrives before relying on a trigger.
• One place to rotate. When a key changes, you update the credential on the source and the investigation setup stays the same.

When something else fits

• If the system can call Tares instead of Tares calling it, the webhook connector fits better, because there is no polled credential to store.
• If the error data sits in a table rather than behind an API, use the Postgres connector with a database user that can read only the table you poll.

Related