How do I trace a LlamaIndex agent deployed to a FastAPI service?

Rius traces a LlamaIndex agent in a FastAPI service once you call rius.init at startup, before you build your indexes and query engines. In Python, a long-running server needs no flush call, because batches export continuously and the exit hook drains the rest on a clean shutdown. Set a stable service name, and call client.shutdown() on graceful stop so Rius can tell a clean exit from a crash.

How it works

The LlamaIndex integration is Python only. With a global init(), Rius turns on every installed integration, so the llama-index instrumentation attaches without extra code. A query becomes a nested trace: retrieval steps as retriever spans, embedding calls as embedding spans, and synthesis model calls as model-call spans with model, token usage and messages. Rius computes cost from those spans, so you never send it.

Spans are queued in memory and exported in batches from a background thread. Request handling never waits on the network, and exporter errors never reach your code.

For a FastAPI service, three things matter:

• Where init runs. Call it once at startup, in the FastAPI lifespan, before the index and query engine exist. The integration patches LlamaIndex process-wide, so it must be in place first.
• Flushing. A long-running server needs nothing in Python. If your server forks workers (gunicorn, multiprocessing), each worker re-creates the exporter thread automatically. Only a hard kill of a worker can lose its last batch.
• Naming. The agent is grouped under its service name, or under RIUS_AGENT_NAME if you set it.

If you also installed the openai extra, a model call made through the OpenAI client appears as a second model-call span under the LlamaIndex span. With glassflow-rius 1.1.1 or later, it is counted once in tokens and cost.

Set it up

1. Install the instrumentation and LlamaIndex. The extra installs the instrumentation, not llama-index itself. Without llama-index in the environment, init() logs a DependencyConflict and the integration stays off. The LlamaIndex integration page covers setup in full.

   pip install "glassflow-rius[llama-index]" llama-index

2. Call init once in the FastAPI lifespan, before you build indexes and query engines, and call shutdown() after the yield. Replace build_query_engine with your own function.

   from contextlib import asynccontextmanager
    
   from fastapi import FastAPI
   import rius
    
   @asynccontextmanager
   async def lifespan(app: FastAPI):
       client = rius.init(api_key="ri_...", service_name="support-agent")
       app.state.query_engine = build_query_engine()  # your own function
       yield
       client.shutdown()
    
   app = FastAPI(lifespan=lifespan)

3. Name the agent. service_name sets the service name. The Agents page groups by RIUS_AGENT_NAME, which defaults to the service name, so set it only if you want a different grouping. RIUS_MAIN_AGENT_VERSION sets the version the Agents page compares, so bump it on each deploy.

   export RIUS_AGENT_NAME="support-agent"
   export RIUS_MAIN_AGENT_VERSION="1.4.0"

4. Know what shutdown() does. On a graceful stop it drains pending spans, stops the exporter thread and sends the final stopped heartbeat. The SDK installs no signal handlers. An unhandled SIGTERM or a SIGKILL sends no stopped ping, and the instance reports gone. That is the crash signal. If workers are killed hard, call client.flush(timeout_millis=30_000) at that boundary to avoid losing the last batch.

5. Keep flush() out of request handlers. It blocks until the export completes, which is the stall the background exporter avoids. The reliability page covers batching, retries and shutdown.

6. Verify. Send one request that runs a query, then open the trace in the console. You see the pipeline as nested spans, with the model-call span carrying model, tokens and cost.

What you get

• A nested trace for each query your FastAPI service answers, from retrieval through embedding to the synthesis model call, with tokens and a list-price cost estimate in US dollars.
• No added latency on the request path, since export runs in the background.
• The agent under one stable name, with versions you can compare across deploys.
• A clean stopped signal on graceful shutdown. With the heartbeat on and an API key set, each worker process shows as its own instance. A crash shows as gone.
• Retrieved document content and messages are content attributes. Document ids and scores stay visible even if you strip content.

When something else fits

• LlamaIndex.TS has no Rius integration. Trace the provider calls with the OpenAI integration and the pipeline around them with manual generations.
• If the same code runs in a serverless function instead of a long-running server, call client.flush() at the end of every invocation and turn the heartbeat off with heartbeat=False.
• For other frameworks and clients, see Integrations.

Related