How do I keep the credentials for JSON API endpoints safe when an AI SRE looks into 5xx errors on an API?

Tares keeps the credentials on the source, so an AI SRE investigating 5xx errors reads events and never the connection details. For a JSON API endpoint, the http_poll connector stores one credential as a secret and never returns it through the API or exports it. For a Postgres table, the connection string is stored as a secret in the same way.

How it works

Each source in Tares holds its own connection details. The agent that investigates a spike in 5xx errors works from the events the source produced, such as the summary line, the labels and the stored payload. It does not receive the URL, the header value or the connection string.

JSON API endpoints (http_poll)

• Tares polls the endpoint itself on the source’s schedule and stores one event per item in the reply.
• One credential travels in one header. credential holds the whole value and is stored as a secret, never returned by the API and never exported.
• credential_header names the header. It is Authorization by default, and you can set a name such as X-API-Key.
• Headers the API wants on every request but that are not sensitive go in headers, which is not secret. Keep sensitive values out of it.

Postgres

• The dsn field takes the connection string, in the form postgresql://user:pass@host:port/dbname.
• The dsn is stored as a secret. It is redacted in the API and omitted from catalog exports.
• Tares reads new and changed rows incrementally, so you can bring in a table where your application records failed requests. See the Postgres connector page for the cursor and labels.

When the API itself returns 5xx

The http_poll connector backs off on a 429 or any 5xx. It makes no request until the wait is over, and the wait doubles from the poll interval up to an hour. The source’s health shows the next try time. Any other 4xx appears as a config error on the source’s health, with the status and a slice of the reply. A wrong credential therefore shows up as a source health problem and not as missing data. The backoff clears on the next good poll.

Set it up

1. Decide which endpoint reports the 5xx picture, for example a status or readings endpoint of your API. If your application records failed requests in a table, use the Postgres connector for that table as well.
2. Open Sources → Add source → HTTP API (poll). Fill in the method, the URL and the credential. Enter one header name and value, which Tares stores as a secret. Add any non-secret headers separately.
3. Press Discover. Tares calls the API once, shows the reply and fills in what becomes an event. Pick the entity key from the fields it saw, then Create.
4. Or declare the source yourself, or send the same object with POST /api/sources. This example sends an API key in the X-API-Key header:

config:
  url: https://api.example.com/v1/readings
  credential: abc123                   # an API key
  credential_header: X-API-Key

5. Set payload_key to the part of the reply that holds the items, id_field to deduplicate repeated reads, and text_template for a one-line summary per event. Nested fields are reachable by dotted name. Declare a number label on the error count or status field so a trigger can watch it.
6. For a Postgres table, open Sources → Add source → Postgres, enter the Connection URL (dsn) and press Discover. Pick the table, confirm the cursor column and save.
7. Open the source and check Recent events and Fields. You see each poll’s events and every field seen, and the credential never appears there. Then ask a connected agent, or use the guided setup described in the AI SRE guide, to investigate the 5xx errors from those events.

For the full field list, see the HTTP API (poll) connector page.

What you get

• The credential lives in one place, on the source, and is stored as a secret.
• The agent investigates from stored events and labels, so it never needs the key or the connection string.
• The API never returns the credential, and catalog exports leave it out, so sharing a source setup does not share the secret.
• Rate limits and failures are handled for you. The connector backs off on a 429 or any 5xx and reports the reason and the next try time in the source’s health.
• A bad credential is visible as a config error on the source, with the status and a slice of the reply, so you can fix it quickly.
• The payload is stored as received, so labels you declare later can still reach every field.

When something else fits

• If the system calls Tares, such as an alerting tool posting to a URL, use the webhook connector instead of polling.
• If your API needs more than one secret header, the connector carries one credential per source. Put the non-secret headers in headers, and keep the secret to one header.

Related