Reliability

The design promise of the Rius SDK: tracing never blocks and never breaks your application. This page explains the machinery behind that and the knobs you can tune.

The export pipeline

Spans are queued in-process and exported in batches from a background thread (OpenTelemetry’s BatchSpanProcessor with an OTLP/HTTP exporter). Span creation is an in-memory operation; the network never sits on your agent’s hot path.

Standard OpenTelemetry environment variables tune the batching:

Variable	Default	Meaning
`OTEL_BSP_MAX_QUEUE_SIZE`	`2048`	Spans buffered in memory. When full, new spans are dropped with a warning.
`OTEL_BSP_SCHEDULE_DELAY`	`5000` (ms)	How often a batch is flushed.
`OTEL_BSP_MAX_EXPORT_BATCH_SIZE`	`512`	Spans per HTTP request.
`OTEL_BSP_EXPORT_TIMEOUT`	`30000` (ms)	Timeout per export attempt.

For very chatty agents, raise the queue size before lowering the sample rate; for latency-sensitive shutdowns, lower the schedule delay.

Retries and failure behavior

Transient failures (connection errors, HTTP 429, 5xx) are retried with exponential backoff and jitter, bounded by the export timeout.
Non-transient failures (other 4xx) are not retried; the batch is dropped and the error logged.
If the backend stays unreachable, spans are dropped and errors are logged. Your application continues normally; exporter exceptions never propagate into application code.

Flushing and shutdown

Pending spans are flushed automatically at interpreter exit. For short-lived scripts and batch jobs, or before a hard kill, flush explicitly:

Python


client = rius.init(...)
...
client.flush(timeout_millis=30_000)   # force an export
client.shutdown()                     # drain, stop the thread, release init()

Since 0.9.0, flush() reports delivery, not just queue drain: it returns False on timeout or when the most recent export was rejected (for example a 401 on a bad API key), so True means your spans actually left. Earlier releases returned True whenever the queue drained, even while every batch was failing.

shutdown() also releases the global init() slot, which is what allows a later init() to reconfigure the SDK.

The TypeScript pipeline is the same BatchSpanProcessor driven by a background timer rather than a thread, tuned by the same OTEL_BSP_* variables, and it retries transient failures the same way. Two differences matter in practice: it installs no exit hook, so nothing is flushed for you, and only the first export failure in a process is logged, naming RIUS_API_KEY and RIUS_ENDPOINT as the likely cause. flush() resolving false is how you detect later failures.

Short-lived processes

Export happens on a background thread, so what you need to do depends on how your process ends:

Where your code runs	What to do	Why
Long-running server (FastAPI, worker, …)	Nothing	Batches export continuously; the exit hook drains the rest on clean shutdown.
CLI or batch job	`client.flush()` before exiting (or rely on the exit hook)	Short processes can finish before the first scheduled batch export fires.
Serverless (Lambda, Cloud Functions, …)	`client.flush()` at the end of every invocation	The runtime freezes the process between invocations; a frozen background thread exports nothing, and the exit hook may never run.
Fork-based workers (gunicorn, multiprocessing)	Nothing at fork time; `flush()` only if workers are killed hard	The exporter re-creates its background thread in each forked child automatically; only an unclean worker shutdown can lose the last batch.

Never call flush() inside request handlers or other hot paths: it blocks until the export completes, which is exactly the stall the background exporter exists to avoid. Flush at process or invocation boundaries only.

Disabled mode

disabled=True (or RIUS_DISABLED=1) attaches no exporter at all: spans are created and dropped in-process. Instrumented code runs unchanged, which makes it the right kill switch for tests and incident response.

Heartbeat delivery

With the heartbeat enabled (the default in both SDKs), the SDK pings the platform for the process’s whole lifetime, completely independent of the trace pipeline above. Python uses a daemon thread, TypeScript an unreferenced timer, and neither keeps your process alive. The delivery rules are deliberately the opposite of the exporter’s:

Never queued, never retried. Liveness is only true fresh; delivering a heartbeat late would be misinformation. A failed ping is dropped, the next interval tries again, and delivery problems warn once and then stay quiet. Like everything in the SDK, failures never raise into your code.
The first ping is immediate, so the agent appears in the Agents view at init() rather than one interval later.
A clean stop means calling shutdown(). That sends the final stopped ping, which is how the platform tells a clean exit from a crash. Both SDKs also send it when the process ends naturally, and neither installs signal handlers, because a library must not own process signals. An unhandled SIGTERM or a SIGKILL therefore sends no stopped ping and the instance reports gone. That is the design: gone is the crash signal. In TypeScript the same applies to any abrupt exit, including process.exit(), so call client.shutdown() if you want the distinction to show up.
Each process has its own identity. In Python a forked child re-arms the heartbeat with a new instance ID, so a gunicorn app shows one instance per worker. In TypeScript each process that calls init() gets its own instance ID for the same reason: one identity never speaks for two processes.
Serverless freezes the process. Between invocations the runtime suspends it, pings stop, and the instance flaps between alive and stale or gone. Heartbeats are a long-lived-process signal and they are on by default, so set heartbeat=False in Python or heartbeat: false in TypeScript (or RIUS_HEARTBEAT=false) in Lambda-style runtimes.

The payload carries no span content: instance ID, agent name, SDK language and version, the IDs of currently open root traces (capped at 32) plus their true count, and the stopped flag. capture_content and mask have nothing to strip here, but note that trace IDs do leave the process in heartbeats, not only in spans.

Partial spans

A span leaves the process when it ends, so a run that is still going is invisible and a run that crashed exports nothing at all. With partial_spans enabled (partialSpans in TypeScript, off by default in both), every sampled span also exports a snapshot at start. The platform replaces that snapshot with the real span when the span ends, so a snapshot that is never replaced is the durable record of what a crashed agent was doing.

Snapshots carry no content, ever. They are filtered to identity attributes: the span kind, the operation, provider and tool names, and the gen_ai.request.* settings such as the model. This is an allowlist rather than a list of things to remove, so content set by third-party instrumentation cannot ride along either. A snapshot is built once at span start and never rebuilt, so content added later in the span’s life cannot reach it.
They share the export pipeline. Snapshots go through the same batching, retries and masking as finished spans, so nothing about the delivery rules above changes.
partial_spans_delay keeps the volume sane. Most agent spans live milliseconds, and a snapshot for one of those is superseded almost instantly. With a delay set (partialSpansDelay in TypeScript, up to 60 seconds), a snapshot waits and is only sent if the span is still open, so a span that finishes first costs nothing on the wire. Anything worth watching live stays open longer than the delay.
A snapshot is a zero-duration span. It carries the same trace and span IDs, parent, name and start time as the final span, with its end time equal to its start time, which is how the platform matches the two.

Next steps

Troubleshooting: symptom-first diagnosis when traces go missing.
Advanced features: sampling, when the volume rather than the pipeline is the problem.