Your First Queries in Cribl Search

The query bar, step by step

Log searches in Cribl Search are based on Kusto Query Language, or KQL, the language the Write Your First Log Query page uses to let you delve into your logs to discover patterns, identify anomalies and outliers, and create statistical models. The page also points to the Cribl Copilot KQL assistant for generating queries from natural language. Its example scenario casts you as a security operations analyst at a company that relies heavily on cloud infrastructure, working with a Dataset of VPC flow logs for insight into network traffic patterns, potential security threats, and resource utilization in different regions.

A log query is a read-only request expressed in plain text that processes log events and returns results in a tabular or graphical format. It consists of one or more query statements, and the predominant type of query statement is a tabular expression statement, in which both input and output consist of tables or tabular Datasets. Tabular statements hold zero or more operators, each beginning with tabular input and yielding tabular output. Operators are connected by a pipe (|), allowing data to flow from one operator to the next for filtering or manipulation, and that sequential piping makes the query's operator order crucial. Think of a funnel: you begin with a full table, the data is filtered, rearranged, or summarized as it passes each operator, and a refined output is left at the end.

Here is the page's opening example query, copied from the first query page:

dataset="cribl_internal_logs" method=*
| where response_time > 1 and method != 'PATCH'
| summarize cnt=count() by method
| where cnt > 42

This is a single tabular expression statement that references a Dataset called "cribl_internal_logs" and filters the data based on the method column. Rows are filtered by the response_time column's value, and rows with the method "PATCH" are excluded. The query then summarizes the count of rows for each method and keeps only methods with a count greater than 42.

For the hands-on steps, the page switches to an unfamiliar Dataset of VPC flow logs called cribl_search_sample, which ships out of the box with Cribl Search. The task is to examine the structure of the data, and the page has you start by copying this query to your clipboard, as the first query page sets out:

dataset="cribl_search_sample" | limit 100

1. Paste the query into the search box. The page adds a tip: you can autoformat the query text.
2. Notice that the query begins with a reference to the data table, cribl_search_sample. This data is piped into the first and only operator, which then selects 100 arbitrary rows.
3. Run the query by selecting Search or pressing Enter.
4. Review the results. The sample Dataset includes various data columns such as _raw, source, host, dataSource, and datatype.

Runnable Kusto code examples in the docs include a Run in Cribl Search link in the top-right corner of the code block. Selecting it opens the query in your Cribl Search environment with the query loaded and ready to execute, and a Copy button copies the query for pasting into your own search.

The syntax the page teaches

Beyond its opening example on the cribl_internal_logs Dataset, the page's examples run against the cribl_search_sample Dataset of VPC flow logs that ships out of the box with Cribl Search, and the operators it teaches are limit, extract, where, and has, together with the earliest and latest time-range values and the count operator. Its example queries also carry project _time, srcaddr, dstaddr, packets lines, take 10 lines, and summarize count() by srcaddr, dstaddr stages, as its examples show.

The limit operator returns a specific number of arbitrary rows, and the page uses it to familiarize yourself with the columns and types of data in a table. You already saw it in the limit 100 query.

The extract operator handles the unparsed information in the _raw field. To parse the data, the page uses extract together with an appropriate Parser, in this case AWS VPC Flow Logs, copied from the first query page:

dataset="cribl_search_sample" | limit 1000 | extract parser='AWS VPC Flow Logs'

Paste it into the search box and select Search or press Enter. The parsed results contain separate fields for each data point in the _raw field, although the actual data in the rows might differ, because the rows are selected arbitrarily.

The where operator filters results that satisfy a certain condition. The page's first where example, from the first query page, compares an integer column to a minimum value using the numerical operator greater-than (>) so you can see only flows that have a certain number of packets:

dataset="cribl_search_sample"
| extract parser='AWS VPC Flow Logs'
| where packets > 5
| project _time, srcaddr, dstaddr, packets
| take 10

The page notes that all rows returned have packet values greater than 5.

A where filter can also pin down a start time. The page's next query, from the first query page, uses earliest=-30m, which means last 30 minutes, and then summarizes the count of flows by source and destination address:

dataset="cribl_search_sample" earliest=-30m
| extract parser='AWS VPC Flow Logs'
| where packets > 5
| summarize count() by srcaddr, dstaddr

To narrow the flows to those that were accepted, the first query page puts a second condition with the string value "ACCEPT" on the Dataset line itself, alongside the bytes check:

dataset="cribl_search_sample" bytes > 100 and action = "ACCEPT"
| limit 1000

Every record returned from that query has the action "ACCEPT" and bytes greater than 100.

The has operator searches for specific strings in the flow log data. It is a case-insensitive search that matches a full term, and the first query page applies it to a specific interface ID:

dataset="cribl_search_sample"
| extract parser='AWS VPC Flow Logs'
| where packets > 5
| where interface_id has "eni-07e5d76940fbaa5c6"

For wider time windows, earliest and latest go at the top level of the query, with the syntax [+|-]#TimeUnit[@SnapToTimeUnit]. Because time ranges are bounded by two extremes, the page advises limiting the query to events within those two times. Its example, from the first query page, covers the window from 35 minutes ago to 5 minutes ago:

dataset="cribl_search_sample" earliest=-35m latest=-5m
| limit 1000
| extract parser='AWS VPC Flow Logs'
| where bytes > 100

The page's walkthrough of that query also describes focusing on flows with more than 100 bytes and action "ACCEPT" with the where operator, and using the timestats operator to aggregate the data on a daily time span (span=1d) grouped by srcaddr, returning the aggregated statistics of the matching flows for each srcaddr, separated by day.

Finally, the count operator determines the total number of events available within a Dataset; here the first query page counts VPC flow events where bytes are greater than 100:

dataset="cribl_search_sample" 
| limit 1000
| where bytes > 100

The page's walkthrough describes the flow as extracting and parsing the cribl_search_sample data with the AWS VPC Flow Logs Parser, filtering for bytes greater than 100, and then using the count operator, so the query returns a single record containing that count.

What the docs flag

First, parsing. The extract parser line in the page's examples is superfluous, because Cribl Search applies the parser automatically; the page keeps the line in its where example for illustration.

Rows you explore with limit are arbitrary, and the page says the same of the extract example: the actual data in the rows might differ, because the rows are selected arbitrarily.

Next, two operator behaviors. The has operator is a case-insensitive search that matches a full term, which is why the page reaches for it for specific string values such as an interface ID. And because time ranges are bounded by two extremes, it is most efficient to limit the query to events within those two times with earliest and latest.

Order, finally, is part of the syntax. Operators are connected by pipes so data flows from one operator to the next, and the page calls the query's operator order crucial.

Frequently asked questions

Which operators does the Cribl Search first query page teach?

The page walks through the limit, extract, where, and has operators, the earliest and latest time-range values, and the count operator. Beyond the opening example's cribl_internal_logs Dataset, the examples run against the cribl_search_sample Dataset of VPC flow logs that ships out of the box with Cribl Search.

How do Cribl Search queries chain operators together?

Queries start by naming the data table, as in dataset="cribl_search_sample", and connect operators with the pipe character so data flows from one operator to the next. Each operator begins with tabular input and yields tabular output, so the order of the operators in the query is crucial.

Do I need to write extract in my Cribl Search queries?

No, Cribl Search applies the parser automatically, so the extract parser line is superfluous. The page keeps the line in its where example for illustration, using the form extract parser='AWS VPC Flow Logs' with the AWS VPC Flow Logs Parser.

What does the has operator match in Cribl Search?

The has operator is a case-insensitive search that matches a full term. The page uses it to narrow VPC flow logs to a specific interface ID with interface_id has "eni-07e5d76940fbaa5c6".