How do I trace an agent built with LlamaIndex and deployed to a Kubernetes deployment?

Rius traces a LlamaIndex agent in a Kubernetes deployment when you call init once at container startup, before you build your indexes and query engines. Call shutdown when the pod stops, so the last spans leave the process. Give every replica the same agent name, and each pod appears as its own instance of that agent. The LlamaIndex integration is Python only.

How it works

• One init per process. The LlamaIndex instrumentation attaches when init() runs, and every installed integration is turned on automatically. Put the call in your entrypoint module, not in a request handler, so it runs once per container process.
• Spans leave in the background. Rius queues spans in memory and exports them in batches from a background thread, so tracing does not sit on your request path. A pod serving traffic keeps exporting continuously.
• Shutdown is your job in a pod. Pod termination arrives as a SIGTERM. Rius installs no signal handlers, because a library must not own process signals, and Python’s default response to SIGTERM ends the process without running its exit hook. So call client.shutdown() from your own handler to drain the queue and send the final stopped ping.
• Each pod is an instance. Every process that runs init() has its own instance ID. Replicas that share one agent name group under that agent on the Agents view as separate instances. If you run gunicorn inside the pod, each worker shows as one instance.
• Structure and cost come from the framework. A query becomes a nested trace: retrieval steps as retriever spans, embedding calls as embedding spans, and synthesis model calls as model-call spans with tokens and messages. Rius computes cost from the model and token counts. If the openai extra is also installed, a call is still counted once in tokens and cost on glassflow-rius 1.1.1 or later.

Set it up

1. Add the integration to your image. Install the extra together with LlamaIndex, for example in your Dockerfile or requirements file. The extra installs only the instrumentation, so llama-index must be in the environment too. Without it, init() logs a DependencyConflict and the integration stays off.

   pip install "glassflow-rius[llama-index]" llama-index

2. Call init() once at startup, before you build indexes and query engines, and shut the client down on SIGTERM. The service_name is the name your agent is grouped under, so pick one name and use it in every replica. If your server already handles SIGTERM, such as uvicorn, call client.shutdown() in its shutdown hook instead, for example a FastAPI lifespan.

   import signal
   import sys
    
   import rius
    
   client = rius.init(service_name="my-agent")  # reads RIUS_API_KEY from the environment
    
   def on_sigterm(signum, frame):
       client.shutdown()  # drain queued spans, send the stopped heartbeat
       sys.exit(0)
    
   signal.signal(signal.SIGTERM, on_sigterm)
   # build indexes and query engines exactly as before

   RIUS_AGENT_NAME is optional and defaults to the service name. Set RIUS_MAIN_AGENT_VERSION to the version you are rolling out, because the Agents page compares agent versions by it. Keep the heartbeat on, since a deployment is a long-lived process and heartbeats are what give each pod an instance.

3. Deploy with the key in a Secret rather than baked into the image. Store the API key in a Secret named rius with the key api-key, and reference it from the container. Kubernetes sends SIGTERM and waits terminationGracePeriodSeconds (30 by default) before SIGKILL. A batch export can take up to 30 seconds by default, so give the pod enough time. Start the process in exec form so it receives the signal as PID 1.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: my-agent
   spec:
     replicas: 2
     selector:
       matchLabels:
         app: my-agent
     template:
       metadata:
         labels:
           app: my-agent
       spec:
         terminationGracePeriodSeconds: 60
         containers:
           - name: agent
             image: registry.example.com/my-agent:1.4.0
             env:
               - name: RIUS_SERVICE_NAME
                 value: my-agent
               - name: RIUS_API_KEY
                 valueFrom:
                   secretKeyRef:
                     name: rius
                     key: api-key

   Never call flush() inside a request handler, because it blocks until the export completes.

4. Run one query against your index, then open the trace in the console. You should see the pipeline as nested spans, with the model-call span carrying model, tokens and cost. The LlamaIndex page lists exactly what is captured, and Reliability covers batching, retries and shutdown in full.

What you get

• A nested trace for each query: retriever, embedding and model-call spans, with the retriever’s output showing what the model was given to work with.
• Tokens and a list-price cost estimate in US dollars on each model call, computed by Rius.
• One agent on the Agents view for the whole deployment, with one instance per pod. A pod that stops cleanly reports a stopped instance, while one killed without a shutdown reports gone, which is the crash signal.
• Retrieved document content and messages are content attributes, so Rius privacy controls apply to them. Document ids and scores stay visible even with content stripped.

When something else fits

• If your agent is written with LlamaIndex.TS, there is no LlamaIndex integration in TypeScript. The OpenAI integration still traces the provider calls, and manual generations cover the pipeline around them.
• If a pod’s process is short-lived, such as a batch job or a CronJob, call flush() before it exits instead of relying on a long-running shutdown path.
• For other frameworks and clients, see the Integrations support matrix.

Related