What should I trace in a customer support agent?

Rius traces a customer support agent best as one trace per customer turn: an agent root span, with generations, tool spans and retrieval spans under it. Set session.id to the conversation and user.id to the customer on every span. You then see each reply, each lookup and each failure in the context of one conversation and one person.

How it works

A trace is one end-to-end run of your agent. For a support agent, that is one customer message handled from request to reply. Each trace has exactly one root span, and the root’s status and duration are the run’s status and latency. A failed root means a failed turn.

Under the root, trace these things:

• The root span (kind AGENT). Wrap the function that handles a turn. Unnamed, it is called invoke_agent plus the agent name, such as invoke_agent support-agent.
• Generations (kind LLM). Each model call records the requested and response model, provider, token usage, finish reasons and, if you keep content, the input and output messages. Rius computes cost from the model and token counts, so you send token usage and never cost. For streamed replies, a generation also gives you time to first token, the latency your customer feels.
• Tool spans (kind TOOL). Order lookups, refunds, ticket updates and account checks each get a span such as execute_tool search_orders. The tool call id joins a tool span back to the generation that asked for it.
• Retrieval spans (kind RETRIEVER). A knowledge base or help-center search gets its own span, named for the data source, with the number of documents requested. The retrieved documents are recorded as ids and scores, not text.
• Steps (kind CHAIN). Classifying intent, deciding whether to escalate or drafting a summary are generic steps. This is the default kind for a function you wrap with observe.

The session is the conversation. Each turn is its own trace, and session.id ties the turns together, even when a conversation outlives a deploy or moves between replicas. The user is the customer, carried as user.id, which ties together everything that person did across conversations and days. Without a session id, each trace is its own session. Without a user id, spans count as anonymous.

Set it up

1. Install the SDK and initialize it once at startup. With a global init(), every installed integration is turned on automatically, so model calls from a supported client library are traced as generations without extra code.

pip install glassflow-rius

import rius
 
rius.init(
    api_key="ri_...",
    service_name="my-agent",
)

2. Wrap the turn handler as the agent root, and wrap each tool as a TOOL span. The function names here are yours.

import rius
from rius import SpanKind
 
@rius.observe(kind=SpanKind.AGENT)
def handle_turn(message: str) -> str:
    ...
 
@rius.observe(kind=SpanKind.TOOL, tool_name="search_orders")
def search_orders(customer_ref: str) -> list[dict]:
    ...

3. Open a retrieval span around your knowledge base search.

with rius.start_as_current_span(
    kind=SpanKind.RETRIEVER, data_source_id="product-kb", top_k=5
) as obs:
    obs.set_input(query)
    docs = search(query)
    obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs])

4. If you trace model calls by hand instead of through an integration, open a generation and pass token usage.

with rius.start_as_current_generation(
    model="gpt-4o",
    provider="openai",
    model_parameters={"temperature": 0.2, "max_completion_tokens": 512},
) as gen:
    gen.set_input(messages)
    response = client.chat.completions.create(model="gpt-4o", messages=messages)
    gen.set_output(response.choices[0].message.content)
    gen.set_response_model(response.model)
    gen.set_response_id(response.id)
    gen.set_usage(
        input_tokens=response.usage.prompt_tokens,
        output_tokens=response.usage.completion_tokens,
    )
    gen.set_finish_reasons(response.choices[0].finish_reason)

5. Scope the user and the session around each turn, so every span inside carries both. Use your own conversation id and customer id. Do not set a process-wide session id in a server that handles many customers, because their traces would merge into one session.

import rius
 
with rius.user(request.user_id):
    with rius.session(conversation_id):
        handle_turn(message)

6. Send an opaque customer id rather than an email or a name, since the value is stored on every span and shown in the console. Customer messages are content, so decide what leaves your process. Setting RIUS_CAPTURE_CONTENT to false strips prompt and response content at export, and names, timings, models, token counts and statuses still flow.

In TypeScript, the equivalents are observe, withSession and withUser. Give wrapped functions a name, because an anonymous one traces as anonymous. The full API is in Tracing your code.

What you get

• A trace list where each row is one customer turn, with a duration and a status you can filter on.
• A waterfall for each turn that shows which retrieval, tool or model call was slow, and highlights the path from the root to the failing span.
• Cost per call, computed by Rius from the model and token counts as a list-price estimate in US dollars, so you can see what a long conversation costs.
• Sessions that show a whole conversation across turns, and a Users page that shows each customer with their traces, failures, cost and time to first token.
• A base for alerts and monitoring across the agent, as described in Monitor AI agents in production.

The terms used here, such as trace, span, generation, session and user, are defined in Concepts.

When something else fits

• If an agent handles exactly one job per process, such as a CLI run or batch job, set the session once with RIUS_SESSION_ID instead of scoping each call.
• If you only need model calls and have no tools or retrieval, the automatic integrations may be enough, and manual spans are optional.
• If customer text must never leave your systems, turn content capture off and rely on metadata such as timings, models, token counts and statuses.

Related