Quarantine, not crash: what a pipeline should do with bad rowsQuarantine, not crash: what a pipeline should do with bad rows

Two failure modes show up constantly in data loads, and they are opposites.

The first is the pipeline that stops dead on the first bad row. One record has a malformed date, the job throws, and 400,000 good records do not land. Someone gets paged, fixes one cell, and reruns the whole thing.

The second is the pipeline that accepts everything. Nothing ever fails, which feels wonderful until someone notices the revenue figure is wrong and nobody can say when it started.

The third option

Bad rows should be routed, not fatal. Validate each record against the rules that actually matter, let the good ones through, and write the failures to a quarantine table alongside the reason they failed and the batch they came from.

This changes the shape of the problem. The load completes. The good data is available. The bad data is not silently dropped and not silently accepted. It is sitting somewhere specific, with an explanation, waiting to be looked at. And crucially, the count of quarantined rows becomes a number you can watch.

A quarantine table that is always empty tells you the rules are too loose. One that is filling up tells you exactly where to look.

Which rules are worth enforcing

Not every check deserves to divert a row. The ones that generally do:

• Required fields that downstream joins depend on. A missing customer id is not a cosmetic issue.
• Types that must parse. A date that is not a date will fail later, further away, more confusingly.
• Referential checks against dimensions you control. An order for a product that does not exist is a real signal.
• Ranges that are physically impossible. Negative quantities, timestamps in the future, percentages above one hundred.

Checks that are better as warnings than diversions: unusual-but-legal values, optional fields, anything where the business genuinely has messy-but-valid cases. Diverting those trains people to ignore the quarantine, which defeats the point.

Schema drift is its own category

A column disappearing, changing type, or arriving with a new name is not a bad row. It is a bad contract, and it should stop the load rather than quarantine 100% of the records one at a time.

The distinction matters operationally. Row-level failures are a data problem and often self-correcting. Schema drift is a change at the source that someone needs to know about today, because every downstream assumption just became suspect.

What good looks like

You can tell a pipeline has this right by asking one question: when something goes wrong, does the system tell you what, how much, and since when? If the answer is a stack trace, the design is not finished. If the answer is a row count, a reason, and a batch id, it is.