How can an AI SRE tell whether GitHub App events made the errors in OpenTelemetry log records start?
Tares gives an AI SRE both sides of the comparison: GitHub App events that arrive as they happen, and OpenTelemetry log records labeled by service. The agent takes the time of the change event, finds the first error lines after it, and checks that those lines were not already there before it. A change that lands just before the first new error lines is your lead, and the evidence lines in both sources back it up.
How it works
Two sources feed the comparison.
- GitHub App events. The GitHub App delivers pull requests, pushes, reviews, releases and CI runs by webhook as they happen, so an event’s time is close to the time of the change. Each event carries labels such as
repo,action,sha,author,branch,titleandurl. A merged pull request readsmergedinaction, and a push hasevent_typepush withon_default_branch. A delivery GitHub sends twice is stored once. - OpenTelemetry log records. The
otlpsource receives OTLP/HTTP logs, and one source covers every service. Theservicelabel comes fromresourceAttributes.service.name. Other fields are reachable asattributes.<key>for a log record’s attributes andtextfor the log line.
The two sources are keyed differently. GitHub events are keyed by repo, and OTLP records by service. Which repository ships which service is knowledge you give the agent, for example in its instructions.
The agent then works in four moves:
- Read the GitHub events for the repository and note the time of the pull request merge, push or CI run that could be the cause.
- Read the log records for the service and find the first error lines in the window.
- Compare the two times. Errors that start after the change event fit the change. Errors that start before it rule that change out.
- Read the lines from before and after the change and compare them. Messages or attribute values that are new or more frequent after the change point at what the change touched. Values that were already common before it are background.
The AI SRE guide describes the same pattern with deploy events: the agent sees that a deploy landed at one time and errors started shortly after.
Set it up
- Connect the GitHub App. On Tares Cloud, choose Connect GitHub in your workspace settings, pick the organization and repositories, and confirm on GitHub. On a self-hosted Tares, open Settings > GitHub, choose Create GitHub App, then Install on GitHub. Tares creates the
githubsource for you. The GitHub connector page has the permissions and the full event list. - On a local Tares, give GitHub a public address so webhooks can arrive. For example:
cloudflared tunnel --url http://localhost:8787
TARES_PUBLIC_URL=https://<your-tunnel>.trycloudflare.com tares up- Add an
otlpsource with aservicelabel, as described on the OpenTelemetry connector page:
- name: otel
connector: otlp
config:
labels:
- { name: service, field: resourceAttributes.service.name, primary: true }- Point your exporter at Tares:
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8787 # OTLP/HTTP → /v1/{logs,traces,metrics}
# or gRPC (otlp-grpc extra): OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317The exporter sends an Authorization header with a Bearer value holding an ingest-scoped API key, needed only when Tares has auth on.
- Optionally, wake the agent on every merged pull request so it has the change time in hand before errors show up. This trigger on the
githubsource fires once per merged pull request in the repositories you list:
- name: pr-merged
sources: [github]
filters:
- { field: event_type, op: eq, value: pull_request }
- { field: action, op: eq, value: merged }
- { field: repo, op: in, value: [acme/checkout-api, acme/payments] }
condition: { aggregate: count, predicate: "> 0", window: 5m }
cooldown: 5m- In the agent’s instructions, name the repository behind each service and ask it to compare the change time with the first error lines, then compare the lines before and after the change.
What you get
- An answer tied to evidence: the change event with its
sha,authorandurl, next to the first error lines with their timestamps. - A clear way to rule changes out. If errors began before the pull request merged, the agent says so and looks elsewhere.
- Change times you can trust. With the GitHub App, events arrive as they happen rather than on a polling interval.
- One OTLP source for every service, so a new service shows up with its own
servicelabel and no extra setup. - A conclusion the agent can write back for the next investigation to find.
When something else fits
- If you connect GitHub with a personal token instead of the App, events are polled every 2 minutes by default, so the change time is less exact. Pushes, reviews, releases and CI runs only arrive with the App.
- If the suspect change is not in GitHub, such as a config edit or a deploy from another system, send it through a webhook source so the agent has a change time to compare against.
Related
- Build an AI SRE
- How can an AI SRE tell whether GitHub commits and pull requests made the errors in Docker container logs start?
- How do I read Postgres tables next to OpenTelemetry log records when an AI SRE investigates?
- How do I read JSON API endpoints next to OpenTelemetry log records when an AI SRE investigates?
Still stuck? Get help