How do I let an AI SRE look at Postgres tables when investigating a disk filling up?

Tares reads a Postgres table with the postgres connector and puts each new or changed row in the timeline, so an AI SRE can see which tenants or orders were affected next to your telemetry. You point a source at the table, pick a cursor column, and Tares polls it incrementally. For data that lives behind an API instead of a database, the http_poll connector does the same job for any JSON endpoint.

How it works

The postgres connector polls a table and creates one event per new or changed row. A cursor column tracks where the last poll stopped. It must be a column that only ever grows, such as an autoincrement id for inserts or updated_at if rows are also updated. Each poll picks up only what is past the cursor, so the agent sees fresh rows without rereading the table.

An entity column, for example tenant_id, becomes the key through a label marked primary. When a disk starts filling up, the agent can line up the rows written in that window with the metrics and logs from your other sources and see which tenants were involved.

A few details that matter when you configure it:

• Discovery. Given the connection URL, Tares lists the tables it can see, then reads the chosen table’s columns and proposes the cursor and labels.
• Columns. You choose which columns to pull. Empty means every column. The cursor, key and time columns are always included.
• Event time. time_column sets the event time. By default it is the cursor if that is a timestamp, otherwise the ingest time.
• Volume. limit caps the rows fetched per poll.
• Summary line. An optional text_template shapes each row into one line.
• Secrets. The dsn is stored as a secret, redacted in the API and left out of catalog exports.

The prerequisite is a Postgres that the Tares host can reach. The driver ships with Tares, so there is nothing extra to install.

For data that sits behind an API, the http_poll connector polls a JSON endpoint on the source’s schedule and stores one event per item. It is the generic path for APIs no other connector covers, such as a status page, a SaaS export or an endpoint of your own. Nested values are reachable by dotted name in labels and trigger filters.

Set it up

1. Ask a connected agent to do it. The agent calls discover_source to list tables and propose the cursor and labels, then test_source for a dry run that fetches sample rows, then create_source. For example:

   Use tares to ingest the orders table at your connection URL, keyed by tenant_id.

2. Or use the UI. Go to Sources → Add source → Postgres, enter the Connection URL (dsn) and press Discover. Pick a table, review the proposed cursor, entity key and labels, choose the columns to pull, then save.

3. Or declare the source yourself, or send the same object with POST /api/sources:

- name: orders
  connector: postgres
  poll: 10s
  config:
    dsn: postgresql://user:pass@host:5432/appdb
    table: orders
    cursor_column: updated_at
    columns: id,tenant_id,status,amount,updated_at   # empty = every column
    labels:
      - { name: tenant_id, field: tenant_id, primary: true }
      - { name: status, field: status }

4. Verify. Open the source. Recent events shows rows on the next poll. Fields reports each column’s coverage, so you can confirm the key column is reliably present.

5. If the data you need comes from an API, add an http_poll source instead. Set the url, and use payload_key to point at the part of the reply that holds the items. A list becomes one event per entry. Set id_field so an item seen in recent polls is not stored twice, and use text_template for the summary line. If the API needs a credential, put the whole value in credential and name the header in credential_header. It is stored as a secret. The connector refuses a poll under 10 seconds and backs off on a 429 or a 5xx. The full field list is on the HTTP API (poll) page, and the Postgres table page covers the table connector in detail.

For the wider picture of how sources, triggers and agents fit together, see AI SRE.

What you get

• Your application’s source-of-truth rows in the same timeline as your telemetry, so the agent can tie a disk problem to the tenants, orders or statuses involved.
• Incremental reads on a cursor, so each poll brings only new or changed rows.
• Control over what the agent sees: you choose the table and the columns, and an empty column list pulls them all.
• A way to add API data as well. With http_poll, any JSON endpoint becomes events, with dotted field names usable in labels and trigger filters.
• A quick check that it works. Recent events shows what each poll brought in, and Fields shows how reliably each column is present.

When something else fits

• If the data you need is logs or metrics rather than table rows, use a connector built for that, such as Prometheus or docker_logs, and keep Postgres for the business side.
• If the system calls out instead of being polled, the webhook connector fits better than http_poll, since it receives pushes from the sender.

Related