What should I trace in a helpdesk copilot for employees?

Rius traces a helpdesk copilot best as one trace per employee request: an agent span at the root, a generation for each model call, and tool and retrieval spans beneath it. Set the conversation as the session and the employee as the user on every span. This gives you a waterfall of each answer, cost per call, and a view of what happened to each employee.

How it works

A trace is one run of the copilot, so make one trace per employee message. Each span has a kind, and the kind decides how Rius shows and counts it. For a helpdesk copilot, the tree looks like this:

• Root span, kind AGENT. Wrap the function that handles one employee message. Its duration is the answer latency, and a failed root marks a failed run.
• Generations, kind LLM. One per model call, such as drafting the reply or deciding which tool to use. A generation carries the model, token usage and finish reason. Rius computes cost from the model and token counts, so you never send a cost.
• Retrieval spans, kind RETRIEVER. The search over your knowledge base, such as IT or HR policies. Record the data source and how many documents you asked for.
• Tool spans, kind TOOL. Each action the copilot takes, such as resetting a password or opening a ticket. Give each one its tool name.
• Step spans, kind CHAIN. The default kind, for steps that are none of the above, such as checking the employee’s permissions.

Two identifiers tie the traces together. The session is the conversation: each turn is its own trace, and session.id groups the turns. The user is the employee, set as user.id. Use an opaque employee id, not an email, because the value is stored on every span and shown in the console. See Concepts for the data model.

Set it up

1. Install and initialize the SDK once, as in the code below. Installation is covered on the Tracing your code page, which also lists every span option. With a global init(), every installed integration is turned on automatically, so model calls through a supported client library already appear as generations.

2. Wrap the request handler in a session and a user scope, and mark the handler as the root agent span. Wrap each tool you wrote with the TOOL kind and its tool name. Open the knowledge-base search as a retrieval span. The names answer, create_ticket, search and hr-policies are examples; use your own.

import rius
from rius import SpanKind
 
rius.init(api_key="ri_...", service_name="helpdesk-copilot")
 
@rius.observe(kind=SpanKind.TOOL, tool_name="create_ticket")
def create_ticket(summary: str) -> str:
    ...
 
@rius.observe(kind=SpanKind.AGENT)
def answer(question: str) -> str:
    with rius.start_as_current_span(
        kind=SpanKind.RETRIEVER, data_source_id="hr-policies", top_k=5
    ) as obs:
        obs.set_input(question)
        docs = search(question)
        obs.set_retrieved_documents([{"id": d.id, "score": d.score} for d in docs])
    ...
 
def handle_message(conversation_id: str, employee_id: str, message: str) -> str:
    with rius.session(conversation_id), rius.user(employee_id):
        return answer(message)

3. Use kind=SpanKind.TOOL for every tool function you trace yourself. A bare observe makes a CHAIN span, which the Tool kind filter and the Tool Loop alert never see.

4. If you trace a model call by hand instead of using an integration, turn that integration off with an explicit instruments list in rius.init, or the call counts twice. Set the usage on the generation yourself, or it carries no tokens or cost.

5. Set the session and user around each request, never process-wide. A process-wide session in a server that handles many employees merges everyone’s traces into one session, and user.id has no process-wide setting.

What you get

• A waterfall for every answer. You see the retrieval, each model call and each tool call in order, with the path from the root to a failing span highlighted.
• Cost and tokens per call. Cost is a list-price estimate in US dollars, computed by Rius from the model and token counts.
• Conversations in one place. Every turn of an employee’s chat groups under its session id. If the session count equals the trace count, nothing is setting session.id.
• A per-employee view. The Users page shows each employee’s traces, failures, cost and time to first token, so you can answer what happened to a given person.
• Alerts that fit the tree. Because your tools are TOOL spans, the Tool Loop alert can count them when the copilot calls the same tool repeatedly.

For the wider picture of watching agents after launch, read Monitor AI agents in production. If employee questions may hold sensitive text, the SDK’s capture_content setting and mask callable decide what content leaves your process. Names, timings, models and token counts still flow.

When something else fits

• If your copilot is a single model call with no tools or retrieval, auto-instrumented generations plus a root span and the session and user scopes are enough. Add tool and retrieval spans when those steps appear.
• Rius traces, prices and alerts on runs. It does not score answers, run evals or manage prompts, so use a separate tool for those.

Related