How do I alert on input tokens for one service?

Rius lets you alert on input tokens for one service with a custom alert that has a Service filter and a rule on the Input tokens metric. The Service filter limits the alert to spans whose service name matches yours, and the rule sets the threshold. Input tokens are measured per trace, so an average rule reads as the average input tokens per trace in the window, not a total.

How it works

A custom alert is made of optional filters, one or more rules and one rolling window. Filters apply first, before any aggregation, so only spans that match feed the rules.

For this question you need two pieces:

• The filter. Use the Service filter. It matches service.name from your instrumentation. Text filters offer =, ≠, in and not_in, and the builder suggests values from your workspace’s live data. Use = for one service, or in if the same alert should cover a short list of services.
• The metric. Input tokens is a numeric metric. The aggregations on offer are avg, min, max, p50, p95 and p99.

Numeric metrics aggregate per trace. avg(Input tokens) is the average input tokens per trace in the window, and p95(Input tokens) is the 95th percentile across traces. Neither is the workspace total. Time to first token, Context usage and LLM calls are the exceptions: they are measured per LLM call, so their aggregations run over calls. An LLM call is an inference call, the kind of span the trace view badges as LLM. Agent, tool, retrieval and embedding spans never count as LLM calls.

The window is shared by all rules and runs from 5 minutes to 24 hours. Rules combine with AND only. For either-or behavior, create a second alert.

Set it up

1. Open the custom alert builder from Alerts and start a new custom alert.
2. Add a filter: field Service, operator =, and pick your service name from the suggested values. An alert takes up to 10 filters with up to 50 values each.
3. Add a rule: metric Input tokens, an aggregation, a comparison operator and a threshold. Choose avg to catch a general rise in prompt size per trace. Choose p95 or max to catch a few unusually large traces. Comparison operators are <, <=, >, >= and ==.
4. Set the window. A longer window smooths out spikes, and a shorter one reacts faster.
5. Read the plain-language preview. It states exactly when the alert fires, and that sentence becomes the alert’s description on its details page.
6. Attach the channels that should receive firings, and set the Renotify policy to off or every N minutes.

The finished alert reads like this. The threshold and service name are your own values:

FILTERS   Service = checkout-agent          (optional, applied before aggregation)
RULES     avg(Input tokens) > 20000
WINDOW    15 minutes                        (shared by all rules)

You can add more rules to the same alert, up to 10. For example, add count(Errors) > 10 to fire only when large prompts and errors happen together. Both rules must hold over the same window.

The full builder reference is on Custom alerts.

What you get

• An alert scoped to one service, so another service’s prompts never trip it.
• A threshold on input tokens per trace, using the aggregation that matches what you care about: avg for drift, p95 or max for outliers.
• A generated description that spells out the firing condition.
• A firing history on the alert’s details page. Each firing records the observed value of every rule against its threshold, so you can see which condition tripped.
• Delivery through the channels you attach. A custom alert can be edited or deleted later. Deleting it removes its firing history after an explicit confirmation.

When something else fits

• If you care about how full the model’s context window is for each call, use the Context usage metric instead. It is measured per LLM call and compared against a value between 0 and 1.
• If the real concern is spend, use the Cost metric with the same Service filter.
• If you need to watch every service the same way, leave the Service filter off, or use in with several services in one alert.

Related