Lineage path · downstream
PII reaches BI
A column tagged PII must not reach a dashboard through any path, unless it was masked on the way.
pii-reaches-bi
DataHub · MCP · Policy as code
KESTREL
Your rules. Your whole graph. Enforced.
Metadata Tests check one entity at a time. Kestrel checks conditions across the lineage graph — then writes what it finds back into the catalog.
Apache 2.0 · Open source · Runs on OSS DataHub
kestrel scan
$ kestrel scan --live --write VIOLATIONS pii-reaches-bi PASS certified-without-owner HIGH patients.ssn reaches Dashboard 'patient_overview' via a 4-hop path, with no masking step on the way. patients.ssn -> stg_patients.ssn -> patient_encounters.patient_ssn -> encounter_summary.patient_ssn -> patient_overview [OK] tagged column + dashboard [OK] wrote violation record [OK] authored incident document
The originality question, answered first
One entity at a time is the wrong unit.
What a per-entity test can ask
- Does this column have a PII tag?
- Does this table have an owner?
- Is this dataset described?
Useful. Kestrel ships one of these too.
What governance actually asks
- Does this PII column reach a dashboard — through any path?
- Did anything mask it on the way there?
- Does this certified model depend on something stale, four hops up?
Conditions over paths. No per-entity test can state them.
One violation, drawn
A per-entity test can only see one box.
This is a real finding from the sample graph. Every box on it passes on its own. The violation is the shape of the path.
note: hop 4 fell back to table-level lineage — column lineage was not populated there, and the report says so rather than pretending otherwise.
Why nobody catches this
The tag stops. The data doesn't.
Someone tags `ssn` at the source and moves on. Four transforms later the same value is sitting in a Looker explore under a different name, untagged, and every per-entity check reports green.
The rule lives in a doc nobody enforces.
"PII must never reach BI" is written down somewhere. It is not executable, nothing checks it on merge, and the first time anyone verifies it is during an audit.
Certification outlives its owner.
A model gets certified, the owner changes teams, the upstream starts failing its freshness check. The badge stays. Everyone downstream keeps trusting it.
Findings die in a terminal.
A scanner prints a wall of red, someone screenshots it into Slack, and the next person to open the asset in the catalog learns nothing at all.
The four rules
Three deterministic. One that reasons.
The templates are LLM-free on purpose — the core enforcement path cannot flake. The agent is additive, and when it can, it compiles your English into one of these.
Entity condition
Certified, unowned
Anything marked Certified must have an owner. Certification without accountability is a promise nobody made.
certified-without-owner
Lineage path · upstream
Stale feeds live
A certified asset must not depend on a stale or deprecated upstream, anywhere within four hops.
stale-upstream-feeds-live
Agentic
Whatever you can say
Type a rule in English. The agent compiles it into a policy file, or plans its own reads and investigates.
freeform
What lands in DataHub
The graph is richer after a run than before it.
Three layers per violation, in increasing order of usefulness to a human. Read-only tools report; this is the part that makes the catalog remember.
01
Tag
`policy-violation` on the offending column — and on the dashboard the data leaked into. The analyst who opens that dashboard sees the finding without knowing the rule exists.
02
Structured property
The machine-readable record: policy id, severity, source, sink, the full path, timestamp. Queryable by whatever agent runs next.
03
Document
An incident write-up linked to both ends of the path: what rule fired, the hop-by-hop table, the real SQL behind those hops, who owns it, and the fix.
OSS-safeupdate_description is Cloud-only and hidden on OSS DataHub, so nothing here depends on it. Tags, structured properties and documents are the whole surface.
READ
Find the policy's subjects, then trace their lineage over MCP.
EVALUATE
Walk each path and test the condition — including what excuses it.
ACT
Tag, record, document. Draft the PR. Ping the owner.