What should I trace in a document processing pipeline?

Rius traces a document processing pipeline best as one root span per run, with a span for each step or tool and a generation for each model call. Set a session id and a user id around the run so you can find every trace for a job or a person. Rius computes cost from the token counts on the generations, so you never send a cost value.

How it works

A trace is one run, and it has exactly one root span. Its duration is the run’s latency, and a failed root means a failed run. Everything else nests under it as a tree. For a pipeline that ingests, extracts, classifies and summarizes documents, this maps well:

• Root span (kind AGENT): one per document, or one per batch if you process a batch as one unit. Its name, duration and status are what run-level views show.
• Step spans (kind CHAIN, the default): stages such as splitting a document into sections, validating fields or assembling the result.
• Tool spans (kind TOOL): calls to an OCR service, a PDF parser or a database write. A tool span records the tool’s name, so the same tool is easy to find across runs.
• Retrieval spans (kind RETRIEVER): lookups against a reference index or knowledge base. The retrieved document ids and scores are metadata and are kept even when content is stripped.
• Generations (kind LLM): each model call that classifies, extracts or summarizes. A generation carries the model, provider, token usage and finish reason. Rius counts these as your LLM calls and prices them.
• Embedding spans (kind EMBEDDING): embedding computations. Their tokens and cost count toward totals, though they are not counted as LLM calls.

Two identifiers tie runs together. The session id (session.id) groups the traces of one job, and the user id (user.id) identifies the person the run served. The SDK stamps both on every span inside the scope. A trace with no session id counts as its own session, and a span with no user id is anonymous. The concepts page defines each of these terms.

Set it up

1. Install the SDK with pip install glassflow-rius and call rius.init once at startup. The tracing guide covers the full API, and the Monitor AI agents in production guide shows what to do with the traces afterwards.
2. Mark the run as the root with observe and kind AGENT. Mark each tool and step with their own kinds. Open a generation around each model call and pass the token counts from the provider’s response.
3. Open the user and session scopes outside the root function, so the root span carries both ids too.

import rius
from rius import SpanKind
 
rius.init(api_key="ri_...", service_name="document-pipeline")
 
@rius.observe(kind=SpanKind.TOOL)
def extract_text(path: str) -> str:
    ...
 
@rius.observe
def split_sections(text: str) -> list[str]:
    ...
 
@rius.observe(kind=SpanKind.AGENT)
def process_document(path: str) -> str:
    text = extract_text(path)
    sections = split_sections(text)
    with rius.start_as_current_generation(
        model="gpt-4o",
        provider="openai",
    ) as gen:
        messages = [{"role": "user", "content": "Summarize: " + sections[0]}]
        gen.set_input(messages)
        response = client.chat.completions.create(model="gpt-4o", messages=messages)
        gen.set_output(response.choices[0].message.content)
        gen.set_usage(
            input_tokens=response.usage.prompt_tokens,
            output_tokens=response.usage.completion_tokens,
        )
    return response.choices[0].message.content
 
with rius.user(submitted_by), rius.session(job_id):
    process_document(path)

submitted_by, job_id, client and the function names are your own. Use your own job id or document id as the session id, and an opaque user id rather than an email. For a TypeScript pipeline, the same tracing guide shows observe, startAsCurrentGeneration, withSession and withUser.

4. Choose the session scope by how your work is shaped. A multi-step job that spans several traces gets one session id. A batch process that handles exactly one job can set RIUS_SESSION_ID once instead. Do not set a process-wide session in a server that handles many users, because every trace would merge into one session. There is no process-wide user setting, so scope the user around each request.
5. If a single document run is long, turn on partial spans with RIUS_PARTIAL_SPANS. Rius then exports a content-free snapshot when each span starts, so a run still in progress is visible and a crashed run leaves a record.

What you get

• A waterfall for each document that shows which stage took the time and which step failed. The path from the root to the failing span is highlighted.
• Cost and token counts for every model call, priced as list-price estimates in US dollars, so you can see which stage of the pipeline spends the most.
• Time to first token for streamed generations, if you record it.
• Sessions that group the traces of one job, and a Users view that shows each person’s traces, failures and cost.
• Content privacy that you control. Documents often hold sensitive text, and setting RIUS_CAPTURE_CONTENT to false strips prompts and responses at export. Names, timings, kinds, models, token counts and statuses still flow, so cost and timing analytics stay complete.

When something else fits

• If your pipeline is one model call with no steps around it, a single generation under a simple root span is enough, and you can skip the step and tool spans.
• If your framework already emits spans for its own steps, let auto-instrumentation create the generations and add observe only for the root and the stages your code owns. If you install the openai extra, drop the manual generation or leave openai out of instruments, or each call is counted twice.

Related