Multi-Tenant Routing in Cribl Stream

The routing model

Cribl Stream separates routing into two stages: filters select a subset of incoming events, and processing Pipelines transform them. The Routes page says it directly: before incoming events are transformed by a processing Pipeline, Cribl Stream uses a set of filters to first select a subset of events to deliver to the correct Pipeline, and this selection is normally made via Routes.

You reach Routes under a Worker Group, then Routing, then Data Routes (on a Single-instance deployment, from the Home submenu), and configure new ones with the Add Route button. Each filter is a JavaScript-syntax-compatible expression, and the docs give two examples:

true
source=='foo.log' && fieldA=='bar'

Multiple Routes can exist in one deployment, but each Route is associated with only one processing Pipeline or Pack, and each Route must have a unique name. The Route Name field is case-sensitive. Sources and Destinations can also carry their own pre- and post-processing Pipelines or Packs.

Routes are evaluated in their display order, from the first Route in the list to the last, and the docs recommend ending the list with a default Route that acts as a catch-all for events matching no filter. Drag a Route vertically to change which one applies first.

The Build Custom Logic page shows where the same expression fields appear elsewhere: Routes use them to build custom logic to dynamically control where data gets routed, and every Function has a Filter field that accepts JavaScript.

Tagging events per tenant

A multi-tenant layout starts with a filter expression that identifies one tenant's events. Filter expressions must evaluate to either true or false: if the result is true, the data gets processed by that Route or Function, and if the result is false, the data will pass through unchanged. The Filter field is set to true by default, so a newly added Route claims every tenant until you narrow it.

The page's best practices list the failure modes that make tenant conditions misbehave:

• Compare only fields you are sure exist. If a field is undefined, a comparison of its properties always evaluates to false: both employeeID.length > 10 and employeeID.length < 10 are false when employeeID is missing.
• == means equal to, while === means equal value and equal type: 5 == 5 and 5 == "5" are true while 5 === "5" is false. Tenant IDs that mix string and number feeds need that choice.
• Use the ternary operator for conditional values, as in (age >= 18) ? 'adult' : 'minor': a condition on the event, a value for the true case, a value for the false case.

Field names with non-alphanumeric characters need the accessor the page documents. A field named @timestamp, user-agent, or kubernetes.namespace_name is read in expressions as __e['<field-name-here>'], note the single quotes, and referenced elsewhere, such as in an Eval function's field names, as the single-quoted literal '<field-name-here>'.

Filtering separates tenants; value expressions label them. Value expressions in Functions create new fields, modify existing ones, or perform transformations on data. The page's worked use case adds a phoneCompany field that resolves to Apple or Other based on the contents of a phoneType field, using this value expression in an Eval Function:

phoneType.startsWith('iPhone') ? 'Apple' : 'Other'

At the Route level there is also the Auto-parse toggle: with it on, Cribl Stream detects each matching event's Datatype and extracts top-level fields from _raw. It runs after the Route matches and before the Pipeline or Pack processes the event, and it does not modify that Pipeline or Pack.

Test tenant filters before wiring them in. The Advanced mode button inside the Filter expression field opens a modal where you paste your expression and select or enter sample data; for filter expressions, the Output resolves to true when the expression is valid, and Cribl Expressions like C.Lookup and C.Time validate the same way in most cases.

Sending each tenant somewhere

Once a Route's filter claims a tenant's events, the Destination setting decides where they go. You can configure each Route to include a Destination to send events to, after they are processed by the Pipeline or Pack. One Route, one Pipeline, one Destination per tenant is the baseline layout.

Where a static name is not enough, toggling Enable Expression on turns the Destination field into a Destination expression field, where a JavaScript expression is evaluated as the name of the Destination. The page notes this evaluation happens at Route construction time, not per event:

`myDest_${C.logStreamEnv}`

The Final toggle decides whether the tenancy is exclusive. When Final is toggled on, the default, the Route processes matched events and sends them to the appropriate Pipeline or Pack, matched events do not continue to the next Route, and non-matched events do. If a Route drops matching events with Final on, those events do not reach lower Routes. With Final off, the Route still processes its matches, but all events continue to the next Route, and the event sent to the current Route becomes a clone. The Add Clone button adds fields, names and values, to the cloned events sent to this Route's Pipeline.

The docs name two ordering strategies, most-specific first and most-general first, with the goal of minimizing the number of filters and Routes an event gets evaluated against. Without any cloning, the broadest expression goes at the top. With a narrow cloned set, clone it upfront, then consume those clones in the Route immediately after, and clone events as late as possible in the Routes.

Beyond one Destination per Route, two documented mechanisms keep events moving. Output Router Destinations function as meta-destinations that allow you to send data to multiple peer Destinations based on rules, evaluated in order top to bottom, with the first match being the winner. The endRoute bumper at the bottom of the Routing table reminds you that, if no Route has Final toggled on, events will continue to the configured Default Destination, the backstop for unmatched traffic.

What the docs warn about

Both pages carry caveats that matter to a tenant table:

• Unreachable Routes. A Route shows an orange triangle when data can't reach it: a higher Route is enabled, its Final toggle is on, and its Filter expression evaluates to true, for example true or 1 === 1. A randomizing method like Math.random() can trigger that intermittently.
• Duplicate events at the Default Destination. If the configured default is also the Destination of a higher Route, duplicate events will reach it: either toggle Final on for a Route or change the Default Destination, and the bumper links to that configuration.
• Routing does not isolate a Source from backpressure. None of QuickConnect, Routes, or the Output Router isolates a Source from the effects of a blocked Destination. Backpressure behavior is determined by each Destination's configuration, so one blocked receiver can prevent the same data from reaching other Destinations, unless the blocking Destination uses Persistent Queue with available capacity.
• Wildcard order is left to right. Wildcard lists are order-sensitive, so negations must be listed before wildcards to take precedence: !foo*, * keeps all terms except those starting with foo, while *, !foo matches all terms. Wildcards cannot target Cribl Stream internal fields that start with __ (double underscore); specify those individually.
• Lookup files over 10 MB in Advanced Mode. For C.Lookup, lookup files larger than 10 MB are not loaded and will not evaluate, though if the expression is valid, the updated values still apply to routed data.

Frequently asked questions

How does Cribl Stream decide which events a Route processes?

Cribl Stream evaluates Route filter expressions in display order, from the first Route in the list to the last, before pipeline processing. Each expression must evaluate to either true or false: true means the data gets processed by that Route, and false means it passes through unchanged. By default, the Filter field is set to true, so a Route applies to all events until you narrow it.

What does the Final toggle do on a Cribl Stream Route?

The Final toggle controls whether events continue to the Routes below after a Route's filter matches them. With Final on, the default, the Route processes matched events and they do not continue to the next Route, while non-matched events do. With Final off, all events continue downstream and the event sent to the current Route becomes a clone.

How do you send different tenants to different Destinations in Cribl Stream?

Each Route can include a Destination that receives events after they are processed by the Route's Pipeline or Pack. You can also toggle Enable Expression on to enter a JavaScript expression that Cribl Stream evaluates as the name of the Destination, or use an Output Router Destination that sends data to multiple peer Destinations based on rules.

When does Cribl Stream mark a Route as unreachable?

Cribl Stream shows an unreachable warning when, with your current configuration, a Route higher in the stack is enabled, has its Final toggle on, and has a Filter expression that evaluates to true, for example true or 1 === 1. In that state, data can't reach the warned Route, and a randomizing method like Math.random() can trigger the condition intermittently.