Connect · item 15
Data quality gates at intake
Bad data does not become less expensive further down the line. more
Health
What the rules are catching
Active and monitoring rules, and everything they have held, released or rejected.
What the gates test for
Six dimensions. A check that has never fired is shown as never having fired, not as evidence of a clean estate.
What happens on failure
The action is stated on the gate, not decided later.
Gates that have never fired
Reported as a zero, not as a pass.
The gates
Each names the dimension it tests, the expression it evaluates, what happens when it fails, and who owns the failures.
| Gate | Dimension | Applies to | Expression | On failure | Run | Failed | Pass | Held | Owner |
|---|---|---|---|---|---|---|---|---|---|
| DQ-01 Mandatory header fields present High | Completeness | Every unit at the first station of its line all-lines | requester, value, materiality and jurisdiction are all non-empty at intake | Quarantine threshold 99.5% | 16,420 | 67 | 99.6% | 4 | N. Ferreira |
| DQ-12 Configuration item has a named owner Medium | Completeness | Technology change and incident intake function Configuration Management Database | the configuration item referenced has an owning team recorded in the configuration database | Warn threshold 95% | 12,436 | 65 | 99.5% | 7 | J. Petrakis |
| DQ-15 Personal data fields carry a classification Highmonitoring only | Completeness | Any unit whose payload may contain personal data all-lines | each field flagged as person-linked carries a classification label before the unit enters the line | Quarantine threshold 99% | 14,392 | 43 | 99.7% | 2 | S. Bergqvist |
| DQ-16 Research payload provenance recorded Mediummonitoring only | Completeness | Research and development intake function Research Notebook and Experiment Data | experiment payloads name their notebook entry and protocol version | Warn threshold 95% | 9,928 | 0 | 100% | 0 | P. Iyer |
Quarantine queue
5 items match the current filter. more
| Item | Function · line | Gate | Field | Observed | Expected | Source | Age | Status | Owner | Outcome |
|---|---|---|---|---|---|---|---|---|---|---|
| IT3B-1038 Device at ring test DQ-12-Q-013 | Information Technology IT3B | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 8.6 days | Released | J. Petrakis | Corrected at source and released back onto the line with the correction recorded. |
| IT10C-1052 Change record at declare DQ-12-Q-004 | Information Technology IT10C | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 5.3 days | Released | J. Petrakis | Corrected at source and released back onto the line with the correction recorded. |
| IT8C-1018 Pipeline run at request DQ-12-Q-006 | Information Technology IT8C | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 5.3 days | Released | J. Petrakis | Corrected at source and released back onto the line with the correction recorded. |
| IT5A-1025 Network change at auto-remediate known DQ-12-Q-008 | Information Technology IT5A | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 30 hr | Released | J. Petrakis | Corrected at source and released back onto the line with the correction recorded. |
| IT1C-1016 Ticket at track DQ-12-Q-007 | Information Technology IT1C | DQ-12 | the configuration item referenced | empty | the configuration item referenced has an owning team recorded in the configuration database | Configuration Management Database | 28 hr | Released | J. Petrakis | Corrected at source and released back onto the line with the correction recorded. |
Which lines carry the intake burden
Quarantine volume by function, all time.
Actions
What is waiting on a person
Quarantine is a holding pen, not a decision. more
Judge a quarantined item
Held by a rule. Nobody has looked at it yet.
Fix the source behind a rejection
Rejected items keep arriving until the upstream cause changes.
Finish a review already open
Picked up, not concluded.
Promote a monitoring rule to active
The rule watches and reports but does not hold anything.
Operations
What this desk is allowed to start
A surface that only reports is not operable. more
Trigger and bound
This desk can start a rule run across a named batch, and it can move an item into quarantine. more
Live observability
What the record shows right now
What happened to everything the rules have held.
Current distribution
171 items
Is policy and strategy coming to fruition
Whether the written intent is holding here
14 of 16 rules are enforcing. 95 items are held and undecided.
Not holding on the record
The written intent is that no unchecked record reaches a downstream figure. 14 rules enforce that today and 2 only watch. more
The gates, the expressions, the thresholds and the quarantine queue are real mechanisms. more
2 of the 16 gates are set to monitor rather than enforce. They are counted in the check totals but they stop nothing. That is a deliberate state for a gate whose false-positive rate has not yet been characterized, and it is shown rather than hidden behind an overall pass rate.
62 items have been held for more than seventy-two hours, the oldest for 9.1 days. A quarantine queue that ages without clearing is a queue that people have learned to route around, so the age is shown on every row and never averaged away.