How do I trace a data analysis agent built with LangChain?

Rius traces a LangChain data analysis agent after you install the langchain extra and call init, with chains, tools, retrievers and model calls as nested spans. You add a run root, a retrieval span and a span per stage with a few lines of your own. You also set the session and the user for each request.

How it works

The LangChain integration hooks LangChain’s callback system, so anything that runs through it is traced: LCEL chains, agents and LangGraph nodes. Each step becomes one span, typed by what it is:

• Chains are CHAIN spans.
• Tool invocations are TOOL spans.
• Retrievers are RETRIEVER spans.
• Model calls are LLM spans with model, token usage and messages. Rius computes cost from them.

The nesting follows your chain structure, so the trace reads like your execution graph.

A data analysis agent usually does work outside LangChain’s callbacks, and you cover that with the tracing helpers:

• A run root. Wrap the function that handles one question with observe(kind=SpanKind.AGENT), so every stage of the run sits under one span.
• A step per stage. Wrap each stage you write yourself, such as loading data, running the query or summarizing, with observe. Unnamed, a span takes the function’s name, or you pass name.
• Retrieval. When you look up table documentation or schema notes yourself, open a RETRIEVER span with start_as_current_span. Give it data_source_id for the index you search and top_k for how many documents you asked for.

set_retrieved_documents() records document ids and scores, not text. That makes it metadata, and it survives content stripping. Put retrieved text in set_output() if you want it captured as content.

Set it up

1. Install the instrumentation. The extra does not install LangChain itself, so name it for a fresh environment. The integration page has the full setup, and the integrations overview covers the other frameworks.

pip install "glassflow-rius[langchain]" langchain

2. Call init once, then add the run root, a span per stage and a retrieval span. In this example search, load_data, run_chain and summarize are your own functions, and run_chain is where your LangChain chain or agent runs.

import rius
from rius import SpanKind
 
rius.init(api_key="ri_...", service_name="my-agent")
 
def lookup_context(question):
    with rius.start_as_current_span(
        kind=SpanKind.RETRIEVER, data_source_id="table-docs", top_k=5
    ) as obs:
        obs.set_input(question)
        docs = search(question)
        obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs])
    return docs
 
@rius.observe(name="load-data")
def load(question, docs):
    return load_data(question, docs)
 
@rius.observe(name="analysis-run", kind=SpanKind.AGENT)
def run_analysis(question: str) -> str:
    docs = lookup_context(question)
    data = load(question, docs)
    result = run_chain(question, data)
    return summarize(result)

3. Scope the user and the session around each request. Every span started inside the scope carries user.id and session.id, including the spans the LangChain integration creates. Use your own conversation id and user id, and prefer an opaque user id to an email.

with rius.user(request.user_id):
    with rius.session(conversation_id):
        run_analysis(request.question)

4. Run one question and open the trace in the console. You should see the run root, your stages, the retrieval span and the nested chain, with the model call carrying model, tokens and cost.

In TypeScript the same pieces are observe, startAsCurrentSpan, withSession and withUser. Install @arizeai/openinference-instrumentation-langchain and keep @langchain/core in the project, then call init as on the integration page. In a short script, await rius.shutdown() before the process exits so queued spans are exported.

What you get

• One trace per question, with the run root on top, your stages beneath it, and LangChain’s chains, tools, retrievers and model calls nested in the same tree.
• Tokens and an estimated cost on every model call. You never send a cost value, and the LangChain span still shows the run’s total while each call is counted once.
• Retrieval spans that show which source you searched, how many documents you asked for, and the ids and scores that came back.
• Sessions that group the turns of one analysis conversation, and a user for each request, so you can see who the traces belong to.
• Failures recorded on the span with the error type, so a failing query stage groups with other errors.

When something else fits

• If you call the model by hand and not through LangChain, trace those calls as manual generations, described on the tracing page.
• If you use text-completion models (BaseLLM subclasses such as langchain_openai.OpenAI), a call is still counted twice. The integration page links to the cases and which layer to keep.
• If one process serves many users, scope the session and user per request. A process-wide RIUS_SESSION_ID would merge every user’s traces into one session.

Related