Worker Group Topologies in Cribl Stream

What worker groups are for

The Manage Worker Groups page defines a Worker Group as a collection of Worker Nodes that share the same configuration, and says groups facilitate the authoring and management of configuration settings. Data flow is configured per Worker Group: the page lists Routes, Pipelines, Sources, and Destinations, and each group also has separate version control.

In the sidebar, select Worker Groups, then Add Worker Group, and on Cribl.Cloud set the Worker Group type to Hybrid. Then enter a unique name, optionally Enable teleporting to Workers for individual Worker Node UI access, and add a description and tags. Cloning a group copies everything configured on the original: Sources, Pipelines, Packs, Routes, Destinations.

Deleting a group removes access to its Version Control menu and the direct recovery option; the page documents two recovery methods:

• Create a new group with exactly the same name; that restores the group with all of its configuration, and if you committed the deletion, select a previous version in Version Control and confirm with Done.
• Revert the deletion commit via command line: stop the Cribl server, find the deletion commit in the Leader Node's history, revert to the last commit before the group was deleted, restart the server.

The page notes that configuring multiple Worker Groups, or more than 10 Worker Processes, requires a Cribl Stream on-prem deployment with a certain license tier.

Mapping workers to groups

The Map Workers to Worker Groups page describes Mapping Rulesets as what defines how Worker Nodes are distributed among Worker Groups. A ruleset is a list of rules that evaluate a Filter expression on the information Worker Nodes send to the Leader Node, with behavior the page likens to Routes: order matters, the Filter section supports full JavaScript expressions, matching is first-match, and one Worker Node can belong to only one Worker Group.

Only one Mapping Ruleset can be active at a time, and activating one deactivates any other existing rulesets. Rulesets can only be configured when Cribl Stream has started in Leader mode. If you define no ruleset of your own, the built-in default ruleset, with a default rule matching all (true), assigns all Worker Nodes to the default Worker Group, unless the Worker Node itself specifies differently.

The page's example rule, myFirstRule, assigns a Worker Node to Group420 when its host IP address starts with 10.10.42 and the node has more than 6 CPUs, or when the CRIBL_HOME variable contains w0. The mapping page lists the rule's Filter value in its example table:

(conn_ip.startsWith('10.10.42.') && cpus > 6) || env.CRIBL_HOME.match('w0')

Worker Nodes can reach a group via a Mapping Ruleset or from the group in the Worker's payload; the page's priority list orders the sources: group matched by a filter in a Mapping Ruleset, group defined in the Worker Node, then the default group. A new ruleset ID can contain only letters, numbers, underscores, and dashes, while older ruleset IDs might include other characters. You can't create custom Mapping Rulesets for Cribl Stream Workers in Cribl.Cloud.

Bootstrapping workers from the leader

The Bootstrap Workers from Leader page states that Workers can completely provision themselves, directly from the Leader, upon initial boot, so a group of any number of Workers can launch and be fully functional within the cluster, in seconds. The Leader serves a bootstrap API endpoint at /init/install-worker.sh that returns a shell script; run it on a supported machine without Cribl Stream installed and the machine boots provisioned as a Worker Node. The Cribl Stream package is not downloaded until you generate the script via the API and then execute it.

The request format on the bootstrap page is:

GET http://<leader hostname or IP>:9000/init/install-worker.sh?token=<authToken>

Each query parameter is optional per the page's API spec table: token is the Leader Node's authentication token; group names the group the Worker joins and falls back to default if not specified; download_url points at a .tgz package when Worker Nodes cannot download from the Cribl CDN; and tag chooses the group a bootstrapped Worker joins, in conjunction with mapping rules. user, user_group, and install_dir default to cribl, cribl, and /opt/cribl.

All Worker Node hosts must enable ongoing outbound communication to the Leader's port 4200 so the Leader can manage the Workers; while the bootstrap script runs, firewalls on each host must also allow port 443 to https://cdn.cribl.io, port 443 to a Cribl.Cloud Leader, and port 9000 to an on-prem Leader.

For interactive work, the UI's Workers page offers Add/Update Worker Node. Under Linux > Add, a modal pre-builds the script from the install package location (default Cribl CDN), the Leader or Outpost hostname/IP, the Auth token, and the target Group; paste it onto the Worker Node's command line. Where the account lacks root, put sudo immediately before bash (the line ends with | sudo bash -) and append -k on a self signed certificate error. The Update path auto-generates a script that changes an existing Worker's Group assignment, and/or reassigns the Worker to a different Leader; Script type defaults to Environment variable, or CLI if not relying on environment variables.

The bootstrap page's chain execution example pipes the script straight to a shell:

curl http://<leader hostname or IP>:9000/init/install-worker.sh?token=<authToken> | bash -

If Workers cannot reach the Cribl CDN, the bootstrap page's internal web server example points download_url at the package file, with the full Leader URL enclosed in double quotes, otherwise the shell treats & as a background operator:

curl "http://<leader hostname or IP>:9000/init/install-worker.sh?token=<authToken>&download_url=https://<your_internal_webserver>/cribl-2.2.0-4589617e-linux-x64.tgz" | sh -

The offline bootstrap variant publishes the package and its .sha256 or .md5 checksum file on a web server the Worker can reach, in the same directory as the package; MD5 is not an acceptable hashing algorithm when Cribl Stream is running in FIPS mode.

In the page's basic tag example, a ruleset rule for Group420 uses the filter cribl.tags.include('awseast1'), and the Bootstrap Workers from Leader page pairs it with this tagged bootstrap URL:

curl "http://<logstream_leader>:9000/init/install-worker.sh?tag=awseast1&token=<authToken>" | bash -

Topology patterns the docs support

The three pages jointly document a short menu of cluster shapes; each bullet names its page.

• Default single-group cluster. With no rulesets of your own, the mapping page's built-in default ruleset assigns every Worker Node to the default group unless the Worker Node itself specifies differently, and the bootstrap page's group parameter falls back to default when omitted.
• Group selection at bootstrap time. Pass group in the bootstrap URL, or pick a target Group in the UI Add/Update flow; the groups must exist first (deploy page), and the Worker Groups page covers creating and cloning them.
• Filter-driven distribution. One active Mapping Ruleset of ordered JavaScript filters splits a fleet across groups, first-match, one node per group (mapping page).
• Tag-driven assignment. Tags in the bootstrap URL match a tag filter in the active ruleset, so the launcher selects the group through the URL (deploy page).
• Per-group configuration planes. Each group carries its own Routes, Pipelines, Sources, Destinations, and version control (worker-groups page).
• Reassignment and recovery. The Update flow re-points an existing Worker to a new group or Leader (deploy page); the worker-groups page adds the two recovery methods for deleted groups.

What the docs flag

• Only one Mapping Ruleset can be active at a time; activating one deactivates the others (mapping page).
• Mapping Rulesets require Leader mode, and no custom rulesets exist for Cribl Stream Workers in Cribl.Cloud (mapping page).
• Configuring multiple Worker Groups, or more than 10 Worker Processes, requires a certain license tier on an on-prem deployment (worker-groups page).
• A deleted group's Version Control menu is unreachable; recovery is same-name recreation or a command line commit revert (worker-groups page).
• Workers need ongoing outbound reach to the Leader's port 4200; only Red Hat, Ubuntu, CentOS, and Amazon Linux are supported Worker platforms; TLS is not enabled by default (deploy page).
• The bootstrap endpoint returns 200 OK even with an invalid token; 403 Forbidden means the node is not configured as a Leader (deploy page).
• Offline bootstrap needs the checksum file in the same directory as the package, and MD5 is not an acceptable hashing algorithm when Cribl Stream runs in FIPS mode (deploy page).

Frequently asked questions

How many workers can one Cribl Stream Worker Group hold?

Any number is on the table: the bootstrap page says a group of any number of Workers can launch and be fully functional within the cluster in seconds. The Worker Groups page's sizing note is that configuring more than 10 Worker Processes requires a Cribl Stream on-prem deployment with a certain license tier.

Which Worker Group does a Cribl Stream Worker join when no rule matches?

The Worker joins the default Worker Group. If you define no Mapping Rulesets of your own, the built-in default ruleset assigns all Worker Nodes to the default group, unless the Worker Node itself specifies differently.

How do I move an existing Cribl Stream Worker to a different group?

You can auto-generate a script from the UI that updates the existing Worker's Group assignment, and optionally reassign it to a different Leader. Use Add/Update Worker Node under Workers, pick Update, choose the new target group, and run the script on the Worker Node.

Can Cribl Stream Workers bootstrap without access to the Cribl CDN?

Yes, the bootstrap page documents both an in-place workaround and an offline workflow. Point download_url at a web server that hosts the package file, enclosing the full Leader URL in double quotes, or publish the package and its checksum file on a reachable web server in the same directory.