Skip to content
Zizeeria
Menu

News

A review that argued back

Note

Six independent reviewers were pointed at one evening's code and found fifty-seven things. Twelve of them were real, and two had been introduced by earlier fixes.

The service above was written in one session by one author who was also its only reader. A hundred and ten tests passed, the type checker was clean, the linter was clean. None of that is a claim about whether a single statement matches the database schema, because every one of those tests had been written to need no database — which made them fast and made them structurally unable to see the two defects that stopped the service starting and stopped it creating anything.

So the code was reviewed adversarially: several readers, each given one lens, each told to report only what they could point at with a file and a line, and each finding then handed to a skeptic instructed to refute it and to default to refuted when unsure. Of fifty-seven claims, twenty-seven were already fixed or were wrong, ten were genuine forks where the fix means choosing something, and twelve were real defects with a traced path from an input to a wrong answer.

Two of the twelve had been created by previous corrections. A fix for a full bag refusing a swap made every swap fail. A fix meant to make a monitoring gauge honest wrapped the two lines that cannot fail and left uncovered the one that can. That is the shape of most of what was found here, and it has a name worth saying out loud: a check that cannot tell the healthy state from the broken one. A panel that always reads zero. A test that defends a sentence that is false. A counter that freezes on good news.

Every fix was verified the same way. Remove it, watch the test fail, put it back, watch it pass. A test that has never failed has proved nothing.