LensReading which lens this session carries.

Connect · item 15

Data quality gates at intake

Bad data does not become less expensive further down the line.

Connector registry

Health

What the rules are catching

Active and monitoring rules, and everything they have held, released or rejected.

What happens on failure

The action is stated on the gate, not decided later.

Block4 gates
The unit never enters the line.
Quarantine8 gates
Held at intake for a named owner to clear.
Warn4 gates
Enters the line carrying a flag that travels with it.

Gates that have never fired

Reported as a zero, not as a pass.

1 of 16 gates have run without a single failure. That is either a genuinely clean condition or a check that is not reaching the data it was written for. It is listed here so somebody has to decide which.

The gates

Each names the dimension it tests, the expression it evaluates, what happens when it fails, and who owns the failures.

GateDimensionApplies toExpressionOn failureRunFailedPassHeldOwner
DQ-10
Source extract inside its freshness commitment
High
Timeliness
Any unit relying on an inbound feed
all-lines
every source system the unit reads has synchronized within its committed freshness window
Warn
threshold 97%
94515
98.4%
8N. Ferreira
ItemFunction · lineGateFieldObservedExpectedSourceAgeStatusOwnerOutcome
IT2B-1009
Access request at confirm
DQ-10-Q-005
Information Technology
IT2B
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse8.5 daysReleasedN. FerreiraCorrected at source and released back onto the line with the correction recorded.
FI11B-1011
Change at escalate
DQ-10-Q-014
Finance
FI11B
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse7.2 daysReleasedN. FerreiraCorrected at source and released back onto the line with the correction recorded.
IT11A-1015
Cost line at publish
DQ-10-Q-002
Information Technology
IT11A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse5.8 daysRejected at intakeN. FerreiraRejected at intake. The unit never entered the line and the requester was told why.
AD1B-1001
Work order at dispatch
DQ-10-Q-007
Administration
AD1B
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse5.1 daysIn reviewN. Ferreira—
LG3B-1041
Corporate action at officer sign
DQ-10-Q-008
Legal
LG3B
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse4.9 daysRejected at intakeN. FerreiraRejected at intake. The unit never entered the line and the requester was told why.
PR10A-1002
Payment run at remittance advice
DQ-10-Q-013
Procurement & Source-to-Pay
PR10A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse4.3 daysHeldN. Ferreira—
RD1A-1008
Research thread at draft findings
DQ-10-Q-009
Research & Development
RD1A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse4.3 daysIn reviewN. Ferreira—
PR10A-1023
Payment run at optimize discount
DQ-10-Q-006
Procurement & Source-to-Pay
PR10A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse3.9 daysHeldN. Ferreira—
CS1C-1039
Contact at detect surge
DQ-10-Q-001
Customer Service
CS1C
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse3.8 daysIn reviewN. Ferreira—
MK4A-1007
Content asset at fact verify
DQ-10-Q-015
Marketing
MK4A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse2.8 daysHeldN. Ferreira—
SL8B-1014
Partner deal at vet
DQ-10-Q-010
Sales
SL8B
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse2.3 daysHeldN. Ferreira—
MK4A-1034
Content asset at sme review
DQ-10-Q-011
Marketing
MK4A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse45 hrReleasedN. FerreiraCorrected at source and released back onto the line with the correction recorded.
IT4C-1039
Infrastructure change at run backup
DQ-10-Q-003
Information Technology
IT4C
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse28 hrHeldN. Ferreira—
HR7A-1036
Assignment at offer internal
DQ-10-Q-004
Human Resources
HR7A
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse2 hrReleasedN. FerreiraCorrected at source and released back onto the line with the correction recorded.
AD10B-1018
Service request at issue actions
DQ-10-Q-012
Administration
AD10B
DQ-10every source system theoutside windowevery source system the unit reads has synchronized within its committed freshness windowEnterprise Data Warehouse2 hrRejected at intakeN. FerreiraRejected at intake. The unit never entered the line and the requester was told why.

Which lines carry the intake burden

Quarantine volume by function, all time.

Procurement & Source-to-Pay57
Information Technology26
Human Resources22
Supply Chain13
Marketing8
AI GBS & Global Business Services8
Finance7
Administration7
Legal5
Revenue Growth Management4
Sales4
Customer Service4
Engineering3
Research & Development3

Actions

What is waiting on a person

Quarantine is a holding pen, not a decision.

Operations

What this desk is allowed to start

A surface that only reports is not operable.

Trigger and bound

This desk can start a rule run across a named batch, and it can move an item into quarantine.

Live observability

What the record shows right now

What happened to everything the rules have held.

Current distribution

171 items

Released4727%
In review2816%
Quarantined6739%
Rejected2917%

Is policy and strategy coming to fruition

Whether the written intent is holding here

14 of 16 rules are enforcing. 95 items are held and undecided.

Not holding on the record

The written intent is that no unchecked record reaches a downstream figure. 14 rules enforce that today and 2 only watch.

What this page is, and what it is not

The gates, the expressions, the thresholds and the quarantine queue are real mechanisms.

2 of the 16 gates are set to monitor rather than enforce. They are counted in the check totals but they stop nothing. That is a deliberate state for a gate whose false-positive rate has not yet been characterized, and it is shown rather than hidden behind an overall pass rate.

62 items have been held for more than seventy-two hours, the oldest for 9.1 days. A quarantine queue that ages without clearing is a queue that people have learned to route around, so the age is shown on every row and never averaged away.