Skip to Content
Tares

How do I tell an AI SRE that error spikes is a known issue with a documented fix?

Tares lets you attach the runbook for error spikes to the entity it affects, using the reference connector. A reference document is declarative material, not a stream, so Tares always surfaces it when an AI SRE correlates on that entity, whatever the read time window. The agent sees the documented fix next to the live events, even when the last spike was hours ago.

How it works

The reference connector stores documents (json, csv, md or txt) and ties each one to an entity by its labels. Unlike an event connector, it has no poll interval and no stream.

  • The config is the data. Each attachment carries its own name, content, format and labels. Editing the source re-materializes it.
  • One event per document. Each attachment becomes one always-surfaced event.
  • Labels are real Tares labels. The labels you tag a document with can be read and used in triggers.
  • No time window applies. A runbook does not age out of a 15-minute read. When an AI SRE reads the entity the document describes, the document comes back in the timeline.
  • No entity key. Correlation is label-native, so the match is made on the label values you set, such as service and environment.

This is what turns a recurring symptom into a known issue. The agent investigating error spikes on a service reads the live errors and the runbook for that service in the same timeline. The runbook explains what the spike usually means and what the fix is. For the wider picture of how an agent investigates, see AI SRE.

Set it up

  1. Write the runbook for error spikes as a short md, txt, json or csv file. State the symptom, the usual cause and the fix steps in the order to run them.
  2. Decide which entity the issue belongs to, and write down its labels, for example service=tares and environment=production.
  3. In Tares, go to Sources → Add source → Reference documents. Upload the file, tag it with those entity labels, and click Create. To add more documents later, re-open the source and click Upload more.
  4. Or define the source as YAML and send it with POST /api/sources. The labels block declares the label names, so reads can correlate on them. Each attachment provides the values.
- name: runbooks connector: reference config: attachments: - name: restart.md format: md content: | 1. kubectl rollout restart deploy/tares 2. watch the pod come healthy labels: { service: tares, environment: production } labels: - { name: service, field: service } - { name: environment, field: environment }
  1. Name the file after the symptom, for example error-spikes.md, so the title tells the agent what the document is about. Replace the example content with your own fix steps.
  2. Verify. Open the source: Recent events shows one event per document, and Fields shows the labels they carry. Then read the entity the document describes. The runbook comes back in the timeline, even outside the read window.
read({ service: tares })

What you get

  • An AI SRE that finds the documented fix for error spikes whenever it correlates on the entity, without you pasting the runbook into each investigation.
  • A runbook that stays visible however long ago the last spike was, because the read time window does not filter it out.
  • Labels you can read and trigger on, since the tags on each document become real Tares labels.
  • One place to maintain the fix. Edit the source, and the document is re-materialized with the new content.
  • A check you can run yourself: the document appears in Recent events and in the entity’s timeline.

When something else fits

  • If you want Tares to react to the spike itself, rather than only explain it, add a trigger on your error source. The runbook then supplies the context when the agent investigates.
  • If the fix is not stable text, such as a value that changes with every incident, keep it out of a reference document. Reference material is for notes, schemas and runbooks that stay true.

Still stuck? Get help

Last updated on