Connect · item 15
Data quality gates at intake
Bad data does not become less expensive further down the line. more
Health
What the rules are catching
Active and monitoring rules, and everything they have held, released or rejected.
What the gates test for
Six dimensions. A check that has never fired is shown as never having fired, not as evidence of a clean estate.
What happens on failure
The action is stated on the gate, not decided later.
Gates that have never fired
Reported as a zero, not as a pass.
The gates
Each names the dimension it tests, the expression it evaluates, what happens when it fails, and who owns the failures.
| Gate | Dimension | Applies to | Expression | On failure | Run | Failed | Pass | Held | Owner |
|---|---|---|---|---|---|---|---|---|---|
| DQ-01 Mandatory header fields present High | Completeness | Every unit at the first station of its line all-lines | requester, value, materiality and jurisdiction are all non-empty at intake | Quarantine threshold 99.5% | 16,420 | 67 | 99.6% | 4 | N. Ferreira |
| DQ-12 Configuration item has a named owner Medium | Completeness | Technology change and incident intake function Configuration Management Database | the configuration item referenced has an owning team recorded in the configuration database | Warn threshold 95% | 12,436 | 65 | 99.5% | 7 | J. Petrakis |
| DQ-15 Personal data fields carry a classification Highmonitoring only | Completeness | Any unit whose payload may contain personal data all-lines | each field flagged as person-linked carries a classification label before the unit enters the line | Quarantine threshold 99% | 14,392 | 43 | 99.7% | 2 | S. Bergqvist |
| DQ-16 Research payload provenance recorded Mediummonitoring only | Completeness | Research and development intake function Research Notebook and Experiment Data | experiment payloads name their notebook entry and protocol version | Warn threshold 95% | 9,928 | 0 | 100% | 0 | P. Iyer |
Quarantine queue
10 items match the current filter. more
| Item | Function · line | Gate | Field | Observed | Expected | Source | Age | Status | Owner | Outcome |
|---|---|---|---|---|---|---|---|---|---|---|
| HR3B-1033 Onboarding case at fast-track pack DQ-01-Q-011 | Human Resources HR3B | DQ-01 | requester, value, materiality and | empty | requester, value, materiality and jurisdiction are all non-empty at intake | Enterprise Data Warehouse | 8.8 days | Held | N. Ferreira | — |
| IT3A-1008 Device at ship DQ-12-Q-003 | Information Technology IT3A | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 8.3 days | Held | J. Petrakis | — |
| IT1B-1018 Ticket at request DQ-15-Q-002 | Information Technology IT1B | DQ-15 | each field flagged as | empty | each field flagged as person-linked carries a classification label before the unit enters the line | Enterprise Data Warehouse | 7.0 days | Held | S. Bergqvist | — |
| IT5C-1010 Network change at install DQ-12-Q-001 | Information Technology IT5C | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 6.8 days | Held | J. Petrakis | — |
| IT4C-1007 Infrastructure change at evidence DQ-12-Q-012 | Information Technology IT4C | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 6.5 days | Held | J. Petrakis | — |
| MK2A-1026 Intelligence brief at assess impact DQ-01-Q-003 | Marketing MK2A | DQ-01 | requester, value, materiality and | empty | requester, value, materiality and jurisdiction are all non-empty at intake | Enterprise Data Warehouse | 6.0 days | Held | N. Ferreira | — |
| IT3B-1004 Device at deploy DQ-12-Q-009 | Information Technology IT3B | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 3.5 days | Held | J. Petrakis | — |
| CS9C-1024 Quality review at score independently DQ-15-Q-004 | Customer Service CS9C | DQ-15 | each field flagged as | empty | each field flagged as person-linked carries a classification label before the unit enters the line | Enterprise Data Warehouse | 3.0 days | Held | S. Bergqvist | — |
| IT8C-1029 Pipeline run at auto-review DQ-12-Q-010 | Information Technology IT8C | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 2.2 days | Held | J. Petrakis | — |
| CS7B-1032 Claim at aggregate defects DQ-01-Q-002 | Customer Service CS7B | DQ-01 | requester, value, materiality and | empty | requester, value, materiality and jurisdiction are all non-empty at intake | Enterprise Data Warehouse | 11 hr | Held | N. Ferreira | — |
Which lines carry the intake burden
Quarantine volume by function, all time.
Actions
What is waiting on a person
Quarantine is a holding pen, not a decision. more
Judge a quarantined item
Held by a rule. Nobody has looked at it yet.
Fix the source behind a rejection
Rejected items keep arriving until the upstream cause changes.
Finish a review already open
Picked up, not concluded.
Promote a monitoring rule to active
The rule watches and reports but does not hold anything.
Operations
What this desk is allowed to start
A surface that only reports is not operable. more
Trigger and bound
This desk can start a rule run across a named batch, and it can move an item into quarantine. more
Live observability
What the record shows right now
What happened to everything the rules have held.
Current distribution
171 items
Is policy and strategy coming to fruition
Whether the written intent is holding here
14 of 16 rules are enforcing. 95 items are held and undecided.
Not holding on the record
The written intent is that no unchecked record reaches a downstream figure. 14 rules enforce that today and 2 only watch. more
The gates, the expressions, the thresholds and the quarantine queue are real mechanisms. more
2 of the 16 gates are set to monitor rather than enforce. They are counted in the check totals but they stop nothing. That is a deliberate state for a gate whose false-positive rate has not yet been characterized, and it is shown rather than hidden behind an overall pass rate.
62 items have been held for more than seventy-two hours, the oldest for 9.1 days. A quarantine queue that ages without clearing is a queue that people have learned to route around, so the age is shown on every row and never averaged away.