How can an AI SRE see which values moved most during 5xx errors on an API?

Tares gives an AI SRE the stats tool, which counts each value of one label in this window against the window before, largest change first. During 5xx errors on an API, the agent sees which values jumped instead of reading the latest few lines of a busy stream. Stats is one of two tools every Tares agent has by default, next to read.

How it works

Every Tares agent has two tools by default, and both go through the same Tares read path:

• stats counts events per value of one label. It compares this window against the window before and lists the values with the largest change first.
• read returns one correlated timeline for an entity.

A busy API stream during a 5xx spike produces far more lines than an agent can read. Reading lines would only show the latest few. Stats answers a different question: which values of a label changed the most compared with the window before. The agent picks one label, such as a service label, and gets a ranked list. The values at the top are where the change concentrates.

Both tools cover the sources of the projects the run is for. If you want the agent to look at other sources, it names them with sources.

The tool is a read. By default a Tares agent reads and concludes, and it does not act. It does not restart, deploy or ticket anything. For the wider picture of how agents investigate, see AI SRE.

Set it up

1. Get the API’s signals into a Tares source. Give the source a label for each field you want to compare, for example service, and mark one label as primary. This example receives OpenTelemetry data and labels it by service:

- name: otel
  connector: otlp
  config:
    labels:
      - { name: service, field: service, primary: true }

2. Start Tares if it is not running yet:

tares up

3. Open or create an agent in a project that includes that source. The agent covers the sources of the projects its run is for, and you can add others with sources. You do not enable stats separately, because it is one of the two default tools.
4. Ask the agent about the incident, for example which values changed most while the API returned 5xx errors. The agent calls stats on one label and gets the values ranked by change, this window against the window before.
5. Let the agent follow up with read on the entity at the top of the list. Read returns one correlated timeline for that entity, so the agent moves from the ranked values to the events behind them.

If you want the agent to act on what it finds, the tools page explains how to register external MCP servers and opt an agent in. Each run then records which external tools it called.

What you get

• A ranked view of which values of a label moved most during the 5xx errors, instead of the latest few lines.
• A comparison built in: every count is this window against the window before, so a value that is always high does not look like the cause.
• A short path from the ranking to the evidence, because read gives the correlated timeline for the entity the agent picks.
• No extra setup for the tools themselves. Stats and read are on for every Tares agent, and they cover the sources of the projects the run is for.
• A safe default. The agent reads and concludes, and it does not restart, deploy or ticket anything unless you connect external tools for that one agent.

When something else fits

• Use read instead of stats when you already know the entity and want its correlated timeline rather than a ranking of label values.
• Connect an external MCP server to an agent when you want it to act, for example to open an issue. Stats only reads, and each external tool call is recorded on the run.
• If the signal you need is not in a source yet, add the source first. Stats can only count values of labels that your sources carry.

Related