How can an agent find which running containers are worth ingesting without me naming them?

Tares lets a connected agent find candidate containers on its own. The agent calls discover_docker, which scans your running containers and proposes one docker_logs source per candidate. For a container you already know, it calls discover_source instead, then runs test_source as a dry run that tails a few lines, and finally create_source.

How it works

The Docker logs connector tails a running container’s logs, one event per line. The agent works through three tools in order:

• discover_docker scans the whole local Docker environment and proposes one source per candidate container. You do not name anything.
• discover_source covers the case where you already know the container and want a proposal for just that one.
• test_source is a dry run. It tails a few lines from the proposed source so you and the agent can see real log output before anything is saved.
• create_source saves the source once the sample looks right.

The Docker logs connector needs access to the Docker daemon. Run taresd on the host, or mount the Docker socket and include the Docker CLI in the image. The connector can also filter lines with optional match and drop regex settings. By default it keeps all lines.

This flow is one part of the wider AI SRE workflow, where agents set up the sources that feed monitoring.

Set it up

1. Connect an agent to Tares and make sure taresd can reach the Docker daemon. If you run Tares in a container, mount the Docker socket and include the Docker CLI in the image.

2. Ask the agent in plain words:

   Use tares to find my running containers and ingest the api-server’s logs.

3. The agent calls discover_docker and gets one proposed source per running container. If you named a container, it can call discover_source for that one instead.

4. The agent calls test_source. This tails a few lines and does not save anything, so you can check that the lines are the ones you want.

5. The agent calls create_source and the source starts polling.

The saved source looks like this. The container value is the name from docker ps:

- name: logs
  connector: docker_logs
  poll: 5s
  config:
    container: tares-demo-api-server   # the container name (docker ps)
    labels:
      - { name: service, const: api-server, primary: true }

If the container runs under Compose without a fixed container_name, set container to the service name and add compose_file, an absolute path or one relative to where the daemon runs, so the tail follows restarts.

You can also do this by hand. In the console, go to Sources → Add source → docker_logs, set container, and use Discover to list the running containers to pick from. Or send POST /api/sources with the same object.

To narrow what gets ingested, add match to keep only lines that match a regex, for example ERROR|WARN, or drop to skip lines, for example HTTP/1.1" to drop access logs. Use label_pattern, a regex with named groups, to turn parts of each line into label context.

To verify, open the source in the console. Recent events shows lines on the next poll. If nothing arrives, check that taresd can run docker logs <container>, because that is the access the connector needs.

What you get

• You do not have to know container names up front. The agent scans for candidates and proposes a source for each one.
• You see real log lines from test_source before anything is saved.
• You end up with a standard docker_logs source, one event per log line, that you can edit later in the console or through the API.
• You can keep noise out with match and drop filters, and keep the tail following restarts for Compose services with compose_file.

When something else fits

• If you already know the exact container, give the name to the agent, which uses discover_source, or add the source by hand with the YAML above.
• If your logs do not live in a local Docker container, another connector fits better. Check the Docker logs page for what it covers, and pick a different connector for logs that come from elsewhere.

Related