Verification
Proving the engine against twenty years of real results.
Unit tests prove the code does what you told it to do. Only real data proves you told it the right thing.
The problem
Race scoring is a rules engine, not arithmetic. Getting it wrong means publishing a result that changes who won a series — and in a sport where results are formally protested, that is not a bug you can ship and patch later.
The engine has to handle six handicap systems, two points systems, fifteen penalty codes from the Racing Rules of Sailing, penalties that can never be discarded, configurable discard profiles, and a tie-break cascade that runs to an appendix of the rules.
I did not want to find out whether it was right by publishing results and waiting for complaints.
The method
- Scraped 1,345 published results files from a public directory of race results — real events, scored by real clubs, using the desktop application the sport has relied on for two decades.
- Parsed each file to recover the original entries, finishes, penalties, handicaps and series configuration.
- Re-scored every race through my engine from those same inputs.
- Diffed every single competitor's points and position against the published result — 144,539 individual comparisons.
The corpus covered 8,424 races, drawn from published results of 99 clubs and class associations across more than ten countries. To be precise about what that number is: it is the breadth of the test data, not a customer list.
The result
99% exact match. Zero bugs found in the scoring engine.
Three mismatches were classed as major. All three were traced to the quality of the source data rather than to engine logic — guest boats inflating the fleet size used in a penalty calculation, and a boat whose handicap rating changed partway through a series.
The remaining variance — around 2,000 rows — was rounding. Corrected times differing by one to three seconds, and bonus points differing by a tenth. None of it changed a finishing position in any race.
What it also found
The interesting result was not about my engine at all. One widely-used code, applied when a boat exceeds the time limit, is not defined anywhere in the Racing Rules of Sailing — and clubs apply it inconsistently. Some score it as the number of starters plus one; others as the last finisher plus one; many with no discernible pattern at all.
I documented that rather than quietly matching whichever behaviour made my numbers look best. Worth noting too: the exercise found no clear defects in the incumbent application either. A verification that only ever flatters the person who ran it is not a verification.
Why this is on my consulting site
Because the objection to AI-assisted delivery is always the same one, and it is a reasonable objection: can you trust the output?
The answer is not to argue about it. The answer is to build the apparatus that would catch you being wrong, run it against reality, and publish what it says — including the parts that are inconvenient. That apparatus is what I bring to an engagement. The code is fast now; the judgement about whether the code is right is the job.