Designing a HIPAA-Ready Log Pipeline with Cribl Stream

PHI in log events

HIPAA, the Health Insurance Portability and Accountability Act, reaches systems that create, receive, maintain, or transmit protected health information (PHI) for a covered entity. PHI is individually identifiable health information, and the framework names the identifiers that mark it: names, addresses, dates, phone numbers, email addresses, Social Security numbers, medical record numbers, health plan beneficiary numbers, device identifiers, and IP addresses or URLs. That last item explains why network metadata in logs is not neutral background for a covered entity.

The Security Rule's controls shape the pipeline: access control limits ePHI to the people and systems that need it, audit control records that access for later review, and minimum necessary pushes every use and disclosure toward the least PHI that will do the job. Log pipelines make each of these easy to fail. One application line can carry a patient name, a chart number, and an IP address, and every forward is a copy. So the pipeline must decide, before events reach their destinations, what to mask, what context to add, and where each class of event goes. The Stream pages cited in this post cover those mechanics: the Mask Function, the Lookup Function, and Routes.

Masking fields before they leave

The Mask Function replaces patterns in events, which the Mask Function documentation calls especially useful for redacting PII and other sensitive data. Patterns run on the JavaScript (ECMAScript) regex engine.

Two defaults matter before you add rules. Filter defaults to true, so the Function evaluates all events, and Apply to fields defaults to _raw. More field names can be added on separate lines, with wildcards and nested addressing. Negated terms are order-sensitive, and double-underscore internal fields match only when enumerated individually.

Each masking rule is a Match Regex paired with a Replace Expression. The match regex supports capture groups and a /g flag that replaces all matches. The replace expression accepts a JavaScript expression or a literal, and capture groups are referenced with a g prefix; g2 names the second group, which lets you keep the key while replacing only the value. The output is a string even when the expression produces a number; the page points to the Numerify Function to fix that.

The page's Social Security numbers example maps directly to PHI: for events whose _raw contains social=#########, it hashes the digits while keeping the key legible. Configuration copied from the page:

Match Regex: (social=)(\d+)
Replace Expression: ${g1}${C.Mask.md5(g2)}

The page's credit card example uses the same technique on values containing hyphens or spaces. Two properties matter here. First, the page states that Cribl does not retain any content that matches the Match Regex that is not inserted into the Replace Expression, so a hash-only replacement leaves the raw value out of the event. Second, C.Mask.md5 is deterministic: the same Social Security number produces the same masked value in every event and destination. Useful for joining, but a linkage property your compliance review should weigh.

Two more controls round out the Function. The advanced Evaluate fields setting adds key-value expression pairs to events in which one or more rules matched, fields the page says are useful in downstream processing and reporting, so a pipeline can flag events that contained PHI. Per-row Enabled toggles disable a rule while retaining its configuration.

Enrichment that does not expose

The Lookup Function documentation describes a Function that enriches events with external fields from lookup tables in CSV, compressed .csv.gz, or binary .mmdb format. Tables are selected from files uploaded via Cribl Stream's UI or referenced by path, including with an environment variable like $CRIBL_HOME/file.csv.

The exposure controls live in the output settings. Output field(s) name the fields added to events after a match and default to all if unspecified, so listing them explicitly keeps the rest of the table out of your events, and a Default Value assigns a string when nothing matches. Add to raw event is toggled off by default; on, it appends looked-up values to _raw as key=value pairs. Keeping it off keeps enrichment in structured fields.

The page documents Exact, Regex, and CIDR match modes for in-memory lookups, a Reload period in seconds that re-checks files updated outside the UI, and Match type for CIDR and Regex modes: First match (the default), Most specific, or All, which returns matches as an array. A Use Disk-Based Lookup toggle switches to disk-backed tables, where key fields must be indexed or the lookup fails. Lookups are case-sensitive by default, with an Ignore case option in Exact mode.

The page's CIDR example assigns a location field by matching destination_ip: one field in, exactly one output field added, nothing touched in raw. The table is copied from the page:

range,location
10.0.0.0/24,San Francisco
10.0.0.0/16,California
10.0.0.0/8,US

The hygiene pattern matters: join a device address to a classification field such as an asset class, and you add context for responders and auditors without patient identifiers. CSV tables are text-based and untyped, so every value arrives as a string, and the table's contents are what lands in your events. Treat the table as data you own and review.

Routing by sensitivity

Routes decide where events go before any Pipeline transforms them. The Routes documentation describes JavaScript-syntax-compatible filter expressions on incoming events, with matches sent to the Route's one Pipeline or Pack. The page's own filter examples:

true
source=='foo.log' && fieldA=='bar'

Two rules shape a sensitivity lane. Each Route is associated with only one Pipeline or Pack, and Routes are evaluated in display order from first to last, so the page recommends a catch-all, typically named default, at the end. The Final toggle, on by default, processes matched events and stops them there; only non-matches continue to lower Routes. With Final off, all events continue and the matched one travels as a clone, tagged with extra fields via Add Clone. The page advises cloning late and consuming a narrow clone immediately after creation.

For a PHI lane, a top Route matching clinical sources or identifier-bearing fields sends the matches to a Pipeline containing the Mask Function, and its Destination points at the restricted store. The Mask page points at the same knob: use the associated Route's Destination field to narrow the Mask Function's scope when unmodified values must reach an archival Destination. With Enable Expression on, Destination becomes a JavaScript expression evaluated at Route construction time; the sample myDest_${C.logStreamEnv} keeps the target name in configuration.

The page also documents failure signals. An unreachable warning, an orange triangle, appears when a higher enabled Route with Final on and a match-all filter makes a lower Route unreachable. The endRoute bumper warns that if no Route has Final on, events fall through to the configured Default Destination, duplicating events when that Destination is also a Route's. Comments above and below Routes are the page's suggestion for a self-documenting table, where an audit of where PHI goes starts. The page is plain about one limit: backpressure belongs to each Destination's configuration, and no routing method isolates a Source from a blocked Destination.

What the docs do not cover

The three pages are mechanics documents, and they stop where HIPAA begins. None references the Privacy Rule, the Security Rule, PHI categories, business associate agreements, or Safe Harbor de-identification. The Mask page does not say whether a hashed identifier meets any de-identification standard. The Lookup page does not discuss who may upload a table or how tables are retained or deleted. The Routes page does not cover role-based access to the Routing UI, retention of delivered events, transport or storage encryption, or an audit log of Route changes. Those omissions describe these pages, not Stream's full feature set. Access control, audit control, minimum necessary, and accounting for where copies of PHI land remain obligations you implement and evidence. No configuration on these three pages is a compliance guarantee.

Frequently asked questions

Can the Cribl Stream Mask Function be set up so PHI values do not remain in events?

Yes. The Mask Function does not retain content that matches a masking rule unless the Replace Expression inserts it back into the event, so a rule that swaps an SSN for its md5 hash leaves only the hash behind. The masked value is written as a string even when the expression produces a number.

Which lookup file formats does the Cribl Stream Lookup Function support?

The Lookup Function reads lookup table files in CSV, compressed .csv.gz, or binary .mmdb format. Tables are selected from files uploaded via Cribl Stream's UI or referenced by path, and in-memory tables can reload automatically on an interval you set.

How do Cribl Stream Routes keep PHI-bearing events on a separate path?

Routes evaluate JavaScript filter expressions in display order and send matching events to a single Pipeline or Pack, with the Final toggle on stopping matched events from continuing to lower Routes. With Final off, matched events travel as clones and all events continue downstream.

Do the Cribl docs on Mask, Lookup, or Routes say a pipeline is HIPAA-compliant?

No. The three pages document masking, lookup, and routing mechanics without HIPAA-specific guidance or compliance guarantees. They do not address retention, access control over the configuration, or whether hashing de-identifies data, and mapping those features to HIPAA obligations is a decision for the covered entity.