Engineering blog Post 08
Honest numbers by construction: building this website
Every company website has an incentive to round up. This one is built so that it cannot do that without failing its own build.
One register for every number
Every figure on the site is an entry in a claims register.
Each entry carries the value exactly as the source states it, the evidence file it came from, and the label a reader needs: live or replay, the model, the number of runs, the 95% range where one exists, and the baseline.
A script writes only the public fields to a data file. Source paths and internal notes never leave the repository. Page copy never contains a figure; it contains a token naming a claim, and the build swaps in the value with its label.
A result that is not verified yet renders as "Benchmark in progress". The data file does not even contain its value:
- id: example.result_not_yet_verified
value: null
status: pending # renders "Benchmark in progress"
Checks that fail the build
After every build, a script reads the generated HTML and fails on any violation. An unknown claim id fails earlier, during the build itself.
The honesty checks cover labels, pairing and pending results. For example, our conjunction result must never appear without its false-manoeuvre rate, lead time and fuel use beside it, or a stated trade-off and a link to the full table.
The hygiene checks cover names and leaks: no prospect names, no links to private repositories, no account identifiers or keys, and no email address except the published contact.
This blog added rules of its own: no internal agent identifiers, and the same checks run on the RSS feed.
The review loop
Changes go through three kinds of review, and every reviewer is an AI agent: independent reviewer agents with no shared context. No outside human has reviewed the site or the code yet. The Product Director agent checks claims and copy against the register.
Two outside-reader agents score the wording. A programme-director agent reviews the result as a visitor would.
That last step was added late, and the next section is why.
The glance test
An AI programme-director review found that the site was honest, accurate and fast, and read like a compliance document. Every fix for earlier feedback had added a sentence: a caveat, a gloss, a note. Pages got longer and greyer.
The standard that replaced it is short.
On overview pages, each section answers "what is the one thing?" in a glance: a heading of eight words or fewer, at most twenty-five visible words, one number or one visual as the focal point, and everything else behind a disclosure.
Caveats stay; they move into notes and disclosures. The release gate is a short screen recording at desktop and phone width.
These posts are articles, not overview pages, so they get longer sections. They still lead with a diagram and a three-line summary.
What went wrong
Accurate is not the same as clear.Link to this quote
Next pass
Every figure on this site has a path back to an evidence file, and the build refuses to ship one that does not. The glance test is the newer rule, and it is why these posts lead with a diagram and a three-line summary.