Skip to Content

How do I trace an agent built with LlamaIndex and deployed to a Docker container?

Rius traces a LlamaIndex agent in a Docker container when you call init once at process start, shut the client down when the container stops, and name the agent with a service name. LlamaIndex tracing is Python only. Query engines, retrievers, embedding calls and model calls arrive as nested spans with tokens and cost. Rius computes the cost itself, so you never send it.

How it works

  • Init runs once, at the entry point. Call rius.init() before you build indexes and query engines. The LlamaIndex integration switches on automatically when its extra is installed, so your pipeline code does not change. See LlamaIndex for the full setup and what lands in a trace.
  • Export runs in the background. Spans are queued in the process and sent in batches from a background thread, so tracing never blocks your agent. The container needs outbound HTTPS access to the default endpoint, https://ingest.eu.console.rius-glassflow.com. If it is unreachable, spans are dropped and your application keeps running.
  • docker stop skips the exit hook. Pending spans are flushed when the interpreter exits normally, but docker stop sends SIGTERM, and Python’s default response ends the process without running that hook. The SDK installs no signal handlers, so turn SIGTERM into a normal exit yourself. Reliability has the full table.
  • The service name is the agent name. The Agents page groups your traces under it. RIUS_AGENT_NAME overrides it and defaults to the service name.
  • Each process is its own instance. With the heartbeat on, which is the default, the agent appears in the Agents view as soon as init() runs, and each process reports under its own instance ID.

Set it up

  1. Add the integration to the image’s Python dependencies. The extra installs the instrumentation, not LlamaIndex, so keep llama-index in the list too:

    pip install "glassflow-rius[llama-index]" llama-index
  2. Set these in the container’s environment instead of in code:

    • RIUS_API_KEY: your API key, which starts with ri_.
    • RIUS_SERVICE_NAME: the name of the agent, for example support-agent. Name it after the role it plays, not after a container ID, so every replica groups under one agent.
    • Optional: RIUS_MAIN_AGENT_VERSION. Set it to the version of the image you ship, and the Agents page can compare two versions of the agent.
  3. Call init() first in your entry point, turn SIGTERM into a normal exit, run the agent, and shut down in a finally block. Start the app in exec form (CMD ["python", "app.py"]) so it is PID 1 and receives the signal:

    import signal import sys import rius client = rius.init() # reads RIUS_API_KEY and RIUS_SERVICE_NAME signal.signal(signal.SIGTERM, lambda *_: sys.exit(0)) # runs the finally block try: # build indexes and query engines exactly as before, then run the agent ... finally: client.flush(timeout_millis=30_000) # force an export client.shutdown() # drain, stop the thread, release init()
  4. Choose the flush that matches how your container ends:

    • Long-running server (FastAPI, a worker): batches export continuously. If the server already handles SIGTERM, such as uvicorn, call client.shutdown() in its shutdown hook, for example a FastAPI lifespan, instead of the finally block. Docker waits 10 seconds before SIGKILL; raise that with --stop-timeout or stop_grace_period in Compose if the final export needs longer.
    • Batch job or CLI: keep the finally block above, because a short process can finish before the first scheduled export.
    • Fork-based workers such as gunicorn: do nothing at fork time. Each child re-creates its exporter thread and shows as its own instance. Flush only if workers are killed hard.
    • Never call flush() inside a request handler. It blocks until the export completes.
  5. Run one query, then open the trace in the console. You should see the pipeline as nested spans, with the model call carrying model, tokens and cost.

For the full install and configuration options, see Integrations.

What you get

  • A nested trace for every query: retriever spans, embedding spans, and model calls with model, token usage and messages. The retriever span’s output shows what the model was given to work with.
  • One agent entry under the name you chose, with a running instance for each container process. Call client.shutdown() on your own exit path and a clean stop shows as stopped. An unhandled SIGTERM or a SIGKILL sends no stopped ping, so that instance reports gone.
  • Cost per call, computed by Rius as a list-price estimate in US dollars.
  • If the openai extra is installed too, the model call also appears as a second span under the LlamaIndex span. On glassflow-rius 1.1.1 or later the call is still counted once in tokens and cost.
  • Content such as retrieved documents and messages is captured by default, and the privacy controls can strip it.

When something else fits

  • If your service is written with LlamaIndex.TS, there is no LlamaIndex integration in TypeScript. The OpenAI integration still traces the provider calls, and manual generations cover the pipeline around them.
  • If the container runs on a Lambda-style runtime that freezes the process between invocations, flush at the end of every invocation and turn the heartbeat off. Reliability explains why.

Still stuck? Get help

Last updated on