Engineering blog Post 08

Honest numbers by construction: building this website

Vivek Raja3 min readPart 8 of 9, published as one series

  • website
  • claims
  • checks
  • review
Every number has one path onto the pagepipeline
Evidencebundlesource fileClaimsregistermode · n · CI ·baselinePublic datainternal fieldsdroppedCopy tokenclaim:idBuildlabelled figureverify-distfail onviolationpending → “Benchmark in progress”, never a figure

Every company website has an incentive to round up. This one is built so that it cannot do that without failing its own build.

One register for every number

Every figure on the site is an entry in a claims register.

Each entry carries the value exactly as the source states it, the evidence file it came from, and the label a reader needs: live or replay, the model, the number of runs, the 95% range where one exists, and the baseline.

A script writes only the public fields to a data file. Source paths and internal notes never leave the repository. Page copy never contains a figure; it contains a token naming a claim, and the build swaps in the value with its label.

A result that is not verified yet renders as "Benchmark in progress". The data file does not even contain its value:

- id: example.result_not_yet_verified
  value: null
  status: pending   # renders "Benchmark in progress"

Checks that fail the build

After every build, a script reads the generated HTML and fails on any violation. An unknown claim id fails earlier, during the build itself.

The honesty checks cover labels, pairing and pending results. For example, our conjunction result must never appear without its false-manoeuvre rate, lead time and fuel use beside it, or a stated trade-off and a link to the full table.

The hygiene checks cover names and leaks: no prospect names, no links to private repositories, no account identifiers or keys, and no email address except the published contact.

This blog added rules of its own: no internal agent identifiers, and the same checks run on the RSS feed.

What the build refuses to shipchecks
HonestyUnresolved claim tokensA figure without its label noteA trade-off shown without its otherhalfA pending claim rendered as a figureA demonstration figure outside acaptionA footnote marker standing in for avalueHygieneProspect names and process termsLinks to private repositoriesAccount identifiers and keysAny email but the contact addressNested links in card gridsMissing title, description, OG image oralt text

The review loop

Changes go through three kinds of review, and every reviewer is an AI agent: independent reviewer agents with no shared context. No outside human has reviewed the site or the code yet. The Product Director agent checks claims and copy against the register.

Two outside-reader agents score the wording. A programme-director agent reviews the result as a visitor would.

That last step was added late, and the next section is why.

The review loop, after it failedworkflow
Draftcopy + claimsBuild checksverify-distAI ProductDirectorclaims and copyAI outsidereaderswordingGlance testdesktop andphoneany fail → back to the draft, remove words before adding any

The glance test

An AI programme-director review found that the site was honest, accurate and fast, and read like a compliance document. Every fix for earlier feedback had added a sentence: a caveat, a gloss, a note. Pages got longer and greyer.

The standard that replaced it is short.

On overview pages, each section answers "what is the one thing?" in a glance: a heading of eight words or fewer, at most twenty-five visible words, one number or one visual as the focal point, and everything else behind a disclosure.

Caveats stay; they move into notes and disclosures. The release gate is a short screen recording at desktop and phone width.

These posts are articles, not overview pages, so they get longer sections. They still lead with a diagram and a three-line summary.

What went wrong

Accurate is not the same as clear.
Link to this quote

Next pass

Every figure on this site has a path back to an evidence file, and the build refuses to ship one that does not. The glance test is the newer rule, and it is why these posts lead with a diagram and a three-line summary.

§ · NEXT PASS

Request a briefing.

A 45-minute session with the evidence behind any figure. Operators can send a week of their queue.

This address is used only to reply to you. The site sets no cookies and loads no trackers.

Briefing requests