How Cribl Stream, Lake, and Search Fit Together

The data path in one diagram-able sentence

No single page documents the full chain, so the diagram-able sentence assembles three documented hops: Cribl Stream delivers events to Cribl Lake, per the Cribl Lake Destination documentation, which notes the destination automatically selects a partitioning scheme that works well with Cribl Search, and Cribl Search queries those Lake Datasets, per the Cribl Search to Cribl Lake documentation. A second route goes straight from Cribl Stream to Cribl Search with the Search destination, per the Cribl Search Destination documentation.

Stream to Lake

The Cribl Lake Destination delivers data to Cribl Lake, per the documentation, and the page lists it as type Non-Streaming, with TLS support, no Persistent Queue support, and available only in Cribl.Cloud.

It receives data from both Cribl-managed Cloud and customer-managed hybrid Stream Worker Groups. The hybrid case is scoped: those Worker Groups must run Cribl Stream version 4.8 or later, and their hosts require outbound HTTP/S access to port 443, with no other port selectable.

Setup runs in the Stream UI: select Products, then Cribl Stream, pick a Worker Group, and add the destination through QuickConnect or Destinations. General Settings holds the Output ID and the Lake dataset, which cannot target the built-in cribl_logs or cribl_metrics datasets.

Backpressure behavior chooses whether to block or drop events when all receivers are exerting backpressure, defaulting to Block. Tags filter and group destinations and are not added to processed events.

The destination requires _time to be a number rather than null; an incoming null or string value is converted to Date.now() / 1000, which the page says creates a readily searchable, time-based partitioning scheme.

Storing data in regular Lake Datasets does not require setting a schema. For full use of Cribl Lakehouse, the page wants events parsed into distinct fields before sending, and it points to the Stream Parser Function so all events have named fields.

If a hybrid Worker Group reports SSL errors, the page points to a firewall or proxy performing SSL inspection: note the Organization ID and AWS Region, swap them into the domain format below, add the exception, and verify outbound port 443 is open. The format is copied from the documentation.

lake-main-<organizationId>.s3.<region>.amazonaws.com
lake-main-goaty-mcgoatface.s3.us-west-1.amazonaws.com

Stream to Search

The Cribl Search Destination, per the Cribl Search Destination documentation, enables Cribl Stream to send logs and native metrics directly to Cribl Search over a dedicated, internal HTTP path. The page lists it as type Streaming, with TLS and Persistent Queue support. Use cases include same-organization and connected-environment Worker Groups, and an ingestion endpoint replacing hand-configured Cribl HTTP Destinations.

The destination serializes events in the same HTTP payload format as Cribl HTTP and delivers them to a Cribl HTTP Source in the environment that hosts Cribl Search, preconfigured to forward all received events into Search's ingestion pipeline. The Send as setting decides whether the destination sends logs, native metrics, or both. Native metric events contain a nonempty __criblMetrics array, and Cribl Search routes them to the metrics store and all other events to log search, without also storing metric events as logs.

Prerequisites are scoped by deployment: Cribl Search must be enabled in a Cribl.Cloud organization with at least one Lakehouse Engine configured to ingest from the Cribl HTTP Source this destination uses. Cross-environment or cross-Workspace paths need both environments on paid plans and eligible for free data transfer via Cribl TCP/HTTP, and Cribl-to-Cribl Auth Tokens on both sides for those paths; same-organization sends use the existing authentication path automatically.

General Settings take an Output ID (-CLONE is appended when cloned), an optional Description, and the Cribl endpoint, which must point to the Address and Port of the corresponding Cribl HTTP Source. Optional Settings include Throttling (default 0, meaning no throttling), Backpressure behavior (default Block), the shared Tags setting, and the Send as choices:

• Logs sends everything to log search, including metric events.
• Metrics routes metric events to the metric store and drops everything else.
• Logs and Metrics routes metric events to the metric store and sends the rest to log search.

Metric routing requires the receiving Cribl Search Source to be enabled for metrics storage; otherwise metric events are discarded rather than stored as logs.

Two planning notes from the same documentation: the receiving lakehouse engine meters ingest after Cribl Stream processes the data, so a Pipeline that parses, enriches, or keeps _raw can show the engine more bytes than the upstream counter, and if Stream processing is not needed, skip this destination and send directly to a lakehouse engine from a Cribl Search native Source.

Search reading Lake

The Cribl Search to Cribl Lake documentation covers the other side: Cribl Lake Datasets work as Cribl Search Datasets out-of-the-box, and Lake-backed Search uses Dataset Type v1 or v2 on a Federated Dataset tied to the Lake Dataset Provider, with Type v2 supporting newline-delimited JSON, Parquet, and DDSS objects.

The provider step needs no action: Cribl Search ships with a preconfigured Dataset Provider for Cribl Lake, cribl_lake. To add a dataset, create the Lake Dataset in the Lake UI first, then in Cribl Search, under Products, Search, Data, Datasets, set kind to Federated Dataset, provider to the Lake provider, and Type to v1 or v2.

To switch an existing Dataset to Type v2, change Type on that Dataset, set a Datatype for each data format, and select Save. The page states Lake Datasets cannot be cloned. A Type v2 Lake Dataset has no Bucket path, Region, Partitioning scheme, or Filter glob. Cribl Search already knows how Cribl Lake lays out and names objects, so a Datatypes table instead pairs a read-only Data Format, JSON, Parquet, or Splunk DDSS, with a Datatype ID that can only parse that format.

Search adds the first row from the Dataset's own storage format: JSON defaults to generic_ndjson, Parquet to cribl_lake_parquet, and Splunk DDSS to splunk_journal. A second row can be added when the Dataset holds both JSON and Parquet formats, which is the maximum, and the row matching the Dataset's own format cannot be deleted.

For Parquet-backed events, the stock cribl_lake_parquet Datatype reads _time from the column named in the timestamp extraction settings, and columns colliding with a reserved field appear with a data_ prefix, so a source column becomes data_source.

The guide also covers the reverse direction: after creating a Dataset in Cribl Lake, you can send Search results to it with the export operator, in this example copied from the documentation:

// export Cribl Search results to an existing Lake Dataset
dataset="cribl_search_sample"
| export to myLakeDataset

One caveat: typically you export data Cribl Search has already parsed, which keeps it Lakehouse-compatible, and unparsed data cannot be searched at Lakehouse speed.

Read speed is conditional: a Lake Dataset linked to a Lakehouse searches significantly faster as long as the query's time range falls within the configured Lakehouse retention period. Older data, still in the Dataset but aged out of the Lakehouse, runs at standard speed, and Lakehouse use can be turned off with this statement from the same page:

set lakehouse="off" dataset="lakehouse_dataset"

What the docs flag

• The Cribl Lake Destination is Cribl.Cloud only, and hybrid Worker Groups must run Cribl Stream version 4.8 or later with outbound port 443.
• The built-in cribl_logs and cribl_metrics datasets cannot be targeted, and an incoming null or string _time is converted to Date.now() / 1000.
• The Cribl Search Destination has no TLS setting of its own for same-organization sends, since that TLS is managed automatically, while cross-environment paths need both environments on paid plans and eligible for free data transfer via Cribl TCP/HTTP, with Auth Tokens on both sides.
• Metric events are not also stored as logs, and are discarded if the receiving Cribl Search Source is not enabled for metrics storage.
• The lakehouse engine meters ingest after Stream processing, S3 Lake Datasets default to the Standard storage class, and Lake Datasets cannot be cloned.

Frequently asked questions

Does Cribl Stream need a schema defined before it can write to Cribl Lake?

No, storing data in regular Cribl Lake Datasets does not require setting a schema. For full use of Cribl Lakehouse, events should arrive parsed into distinct fields, and the docs point to the Stream Parser Function so all events have named fields.

How does the Cribl Search Destination route native metric events?

The Send as setting decides: Logs sends everything to log search including metric events, Metrics routes metric events to the metric store and drops everything else, and Logs and Metrics routes metrics to the metric store and the rest to log search. Metric events carry a nonempty __criblMetrics array and are not also stored as logs.

Do Cribl Lake Datasets need setup in Cribl Search before they can be searched?

They do not. Cribl Lake Datasets work as Cribl Search Datasets out-of-the-box, and Cribl Search comes with a preconfigured Dataset Provider for Cribl Lake called crib_lake. You add a Federated Dataset, set its Dataset Provider to the Lake provider, and set Type to v1 or v2.

Why might a query against a Lake Dataset miss Lakehouse speed?

The fast path only applies when the Dataset is linked to a Lakehouse and the query's time range falls within the configured Lakehouse retention period. Querying older data, still in the Dataset but aged out of the Lakehouse, runs at standard search speed. The guide's set statement example shows how to turn Lakehouse off for a Dataset.