What runbook should an AI SRE follow for a cron job that stopped?

Tares lets you write the runbook for a cron job that stopped as a markdown document and attach it to the service with the reference connector. Because the document carries the service’s labels, Tares surfaces it whenever an agent correlates on that service. The agent gets your steps next to the live events, and the document does not age out of the read window.

How it works

The reference connector holds documents (json, csv, md or txt) that you attach to an entity by its labels. It is declarative reference material, not a stream.

• One event per document. Each attachment becomes one always-surfaced event.
• Matched by labels. You tag each document with the labels of the entity it describes, such as service and environment. Those labels become real Tares labels you can read and trigger on.
• Not limited by the read window. When an agent correlates on that entity, the runbook comes back in the timeline regardless of the read time window. A runbook does not age out of a 15-minute read.
• No poll interval. The config is the data. Editing the source re-materializes it, and there is no entity key because correlation is label-native.

This fits the way an AI SRE works. The agent investigates the service, and your stopped-cron runbook arrives with the evidence instead of living in a wiki the agent never opens.

Set it up

1. Write the runbook. Put the steps you want followed when a scheduled job stops in a markdown file, for example cron-stopped.md. Write them in your own terms: how to check that the scheduler is running on the host, how to find the job’s last successful run in your own logs or scheduler, how to run it by hand once, and who to tell. Keep each step short and in order.
2. Pick the labels. Use the same label values your event sources use for the service that owns the job, for example service and environment. A reader that correlates on those labels then finds the runbook.
3. Add the source. In Tares, open Sources → Add source → Reference documents. Upload the file, tag it with the labels, and click Create. To add more documents later, re-open the source and use Upload more.
4. Or declare it as YAML. The same object works as config, or you can send it with POST /api/sources:

- name: runbooks
  connector: reference
  config:
    attachments:
      - name: cron-stopped.md
        format: md
        content: |
          1. Check that the scheduler is running on the host
          2. Find the job's last successful run in your own logs
          3. Run the job once by hand and watch it finish
          4. Tell the service owner what you found
        labels: { service: nightly-report, environment: production }
    labels:
      - { name: service, field: service }
      - { name: environment, field: environment }

The labels list under config declares the label names so reads can correlate on them. Each attachment supplies the values. 5. Verify. Open the source. Recent events shows one event per document, and Fields shows the labels they carry. Read the entity by its labels and the runbook comes back in the timeline, even outside the read window.

What you get

• Your cron runbook is surfaced every time an agent correlates on the service it belongs to.
• The runbook is available at any read window, so a job that stopped hours ago still brings its steps with it.
• One source can hold runbooks for many services. Each document carries its own labels, so each service gets its own steps.
• Runbooks are plain text you control. Editing the source updates what the agent sees.
• The labels on each document are real Tares labels, so you can read and trigger on them like any other.

When something else fits

• If you need a stream of data about the job itself, such as its logs or the rows it writes, use an event connector. The reference connector holds documents only.
• If your steps change every few minutes or come from another system, keep them there and attach only the stable parts here.

Related