How do I key Postgres rows by tenant_id and pull only some of the columns?

Tares keys Postgres rows by tenant_id when you mark a tenant_id label as primary, and it pulls only the columns you list in the columns setting. Leave columns empty to pull every column. The cursor, key and time columns are always included, so you never lose the fields Tares needs to track and place each row.

How it works

The Postgres connector polls a table incrementally and creates one event per new or changed row. It follows a cursor column that only grows, such as an autoincrement id or updated_at.

Three settings shape what you get:

• Entity key. A label with primary: true becomes the entity key. Mark tenant_id as primary and every row is keyed by its tenant.
• Columns. columns is a comma-separated list. Empty means every column. The cursor, key and time columns are always added, even if you leave them out of the list.
• Event summary. text_template is optional. It is a str.format template over the row, for example '{tenant_id}: {status} ${amount}'.

The database and table values are also available as label fields, whichever columns you pull. The time_column setting sets the event time. By default it is the cursor if the cursor is a timestamp, and ingest time otherwise. The limit setting caps the rows fetched per poll.

The Postgres connector page lists every field. Discovery works from the connection URL: it lists the tables Tares can see, then reads the chosen table’s columns to propose the cursor and labels.

Set it up

1. Make sure a Postgres is reachable from the Tares host. The driver ships with Tares, so you install nothing extra.
2. Add the source. In the UI, go to Sources, Add source, Postgres, enter the Connection URL and select Discover. Pick your table. Tares proposes the cursor, entity key and labels from the columns. Choose which columns to pull, then save. You can also ask a connected agent: use tares to ingest the orders table, keyed by tenant_id.
3. Or declare the source as YAML, or send the same object with POST /api/sources. This example keys rows by tenant_id and pulls five columns:

- name: orders
  connector: postgres
  poll: 10s
  config:
    dsn: postgresql://user:pass@host:5432/appdb
    table: orders
    cursor_column: updated_at
    columns: id,tenant_id,status,amount,updated_at   # empty = every column
    labels:
      - { name: tenant_id, field: tenant_id, primary: true }
      - { name: status, field: status }

4. Optionally add text_template under config to shape the event summary from the row fields.
5. Verify. Open the source. Recent events shows rows on the next poll. Fields reports each column’s coverage, so use it to confirm that tenant_id is reliably present on every row. If coverage for the key column is not complete, rows without it have no tenant key.

Tares stores the dsn as a secret. It is redacted in the API and omitted from catalog exports.

What you get

• Every row is keyed by tenant_id, so each tenant’s rows group under one entity in the timeline.
• Only the columns you list are pulled, plus the cursor, key and time columns that Tares always includes.
• Incremental reads, so each poll brings in only new or changed rows.
• An optional readable summary for each event through text_template.
• A coverage check in Fields that shows whether the key column is present on your rows.
• Your application’s source-of-truth data alongside the rest of the timeline.

When something else fits

• If you want every column, leave columns empty. You still key by tenant_id with the primary label.
• If your data lives behind a JSON API rather than in a table, use the HTTP poll connector instead. It is a separate connector from Postgres.

Related