Home/Methods

Process · Writing

Publishing without invented numbers: how we write these articles

The articles on this site are drafted by AI agents, checked by other agents and then by a script with no model in it. The method rests on one finding from the first round: language models invent quantities in words as readily as in digits, so a checker that only reads digits passes most of what it should stop.

Updated 2026-10-11

What this is for

Our research articles make claims a reader can check. A wrong figure costs more than the article earns. We draft with AI agents because that lets a small team publish in English and Romanian, and the price is a writer that will supply a plausible quantity whenever a sentence seems to want one.

So the method does not rely on the writer behaving. It assumes the writer will invent, and it is built to catch the invention before publication. The same approach applies to anything generated at scale from a set of facts: articles, product copy, social captions, question banks. The examples below come from several of these.

The brief is the only source

Every piece starts with a brief that holds the exact facts and the file each one came from. Writers get the brief and the source files, nothing recalled from memory. The brief also lists what gets a draft rejected: a number without a source, a person's name, an internal identifier, any sum of money, and anything that blurs published with built.

Every number in a draft, and every quantity written as a word, must appear in a facts list attached to the article, paired with its source file. The list is part of the deliverable, not a note for later.

The brief is also where errors start. On another project, a list of "confirmed facts" we handed to generating agents contained mistakes of ours, and the agents repeated them with the same authority as the true ones. It happened four times. Once it produced a quiz question with no correct answer, because the explanation repeated the same inverted figures and nothing inside the text could contradict them. Three independent verifiers caught it only because they read the primary text in full rather than our list.

Since then, facts live in a versioned file with the source and the date we read it. Anything unconfirmed sits in a separate section that writers are told not to use. Verifiers are asked to challenge the list, not only the drafts.

Writers, verifiers, repair

Writers. For the research section, six agents each wrote one article, in both languages, from the brief.

Verifiers. Six more agents each took one draft and checked every figure against the brief. They run with a separate prompt. A writer checking its own draft checks what it meant to say, not what it said.

Repair. Repair agents applied the verifiers' findings. The first round contained 25 invented quantities, and 117 corrections were applied in total.

Repair is not the last step. A rewritten paragraph is new text that nobody has checked, so the repaired draft goes back through the gate described next.

The deterministic gate, and what it missed

The final gate is a short script. It pulls every run of digits out of both language versions and compares each one with the set allowed by the facts list. A number outside the set fails the article. There is no judgement in it, which is the point: it cannot be persuaded.

We expected it to be the strong check. It was not enough on its own. Run on the unrepaired drafts, it passed them too, because the invented quantities in those drafts were written in words: "a few minutes", "three incidents". A digit extractor does not see words.

The agent verifiers caught the quantities in words; the script caught digits the agents read past. Each catches a class of error the other misses, and neither is enough alone. That is why the brief now requires quantities written in words to be in the facts list as well, and why both checks run.

Checks belong on the text that ships

A different pipeline taught us where to put a check. It drafts customer replies through a long chain of steps, several of which regenerate the text with a model. A deterministic corrector for one recurring phrasing ran once, at the start of the chain. At least five later steps rewrote the text. The same error came back three times in a row, each time by a different path, and each time we believed it fixed.

The diagnosis came when we ran the corrector by hand on the final output, and it fixed it. The function was fine; the draft had simply never passed through it. Correctors now run at the common exit, on the text that actually leaves. Ones that are safe to repeat run at both ends. After any step that regenerates text, we ask which checks the new text has skipped.

A style gate runs last. It removes invisible Unicode characters that generated or pasted text can carry, flags a list of phrases that read as machine-written, reports overused dashes and gives a score out of 100, with 80 as the bar for publishing. Flagged phrases are listed, not rewritten automatically. Rewriting them with a model would produce more of the same.

Where we were wrong

We trusted a prompt to prevent a defect. A pipeline that writes social captions from video clips asked the model for the brand's voice. When a clip showed a person introducing themselves by name, the model wrote that introduction as the brand's own words and turned the name into a common noun. It was published.

We did not generalise from one case. A scan of the queue found one such caption in roughly ninety, so the fix is narrow: a deterministic check for a self-introduction with a name, then a rewrite, then rejection if the rewrite still fails. Our first version of that check matched without regard to case, which defeated its own capital-letter pattern, and its first scan reported 23 broken captions that were fine. We now test a guard on good and bad examples before believing what it reports.

The same pipeline also misspelled proper names, systematically rather than by accident. The correction is deterministic: a list of names taken from product titles, with fuzzy matching only on capitalised words, at a similarity of 0.82. Each guard around it was paid for with a real false positive found in a dry run.

Markdown twins and llms.txt

Every article also exists as a Markdown file with front matter, a citation and a CC BY 4.0 licence. The HTML page links to it as an alternate, and llms.txt lists it. Language models and crawlers read plain text more reliably than a styled page, and an article we want cited should be easy to quote exactly.

One trap. The web server did not know the Markdown type and served the file as generic binary data, so browsers downloaded it instead of showing it. It took one rule in the server configuration. The check that proves the fix is the content-type header in the response, not a 200 status.

Limits

The method shows that every number traces to a source file. It does not show that the source is right; that is the job of the facts list, and the facts list can be wrong. It does not catch a true figure used in the wrong context, or a vague claim that contains no quantity at all.

The two gates cover the classes of error we know about. The quantity-in-words class was found by failure, not by design, and we expect there are others we have not met yet.

Checklist

Facts go in a file, with the source and the date read, before anything is written. Writers get the file. Verifiers are separate agents and challenge the list as well as the draft. Repair is followed by another check, never by publication. A script compares every digit with the allowed set; agents look for quantities in words. Correctors run on the final text. The style gate runs last, and flagged phrases are rewritten by hand. The served content type of each Markdown twin is checked, not assumed.

Questions and answers

How do you stop an AI writer from inventing numbers in articles?

Stopping an AI writer from inventing numbers starts with a brief that holds the exact facts and their source files, and a facts list in which every number and every quantity written as a word must appear. Separate verifier agents check each draft against the brief, repairs go back through the checks, and a script compares every digit with the allowed set. The method assumes the writer will invent and is built to catch it.

Why is a script that checks digits not enough to catch invented figures?

A script that checks digits is not enough because language models also invent quantities in words, such as a few minutes or three incidents, and a digit extractor does not see words. In the first round the digit gate passed unrepaired drafts that contained invented quantities. Agent verifiers caught the quantities in words, while the script caught digits the agents read past, so both checks run.

Where should a deterministic text corrector run in an AI pipeline?

A deterministic text corrector should run at the common exit of the pipeline, on the text that actually leaves. When a corrector ran only at the start of a long chain, later steps that regenerated the text with a model brought the same error back repeatedly by different paths. Correctors that are safe to repeat can run at both ends, and after any regenerating step it is worth asking which checks the new text skipped.

Can a list of confirmed facts make AI-generated content less accurate?

A list of confirmed facts can make AI-generated content less accurate when the list itself contains mistakes, because generating agents repeat those mistakes with the same authority as true facts. In one case an inverted figure produced a quiz question with no correct answer. Facts should live in a versioned file with source and date read, unconfirmed items kept apart, and verifiers told to challenge the list.

Why publish Markdown versions of articles alongside HTML pages?

Publishing Markdown versions of articles helps language models and crawlers, which read plain text more reliably than a styled page, so an article meant to be cited can be quoted exactly. Each Markdown twin carries front matter, a citation and a licence, is linked from the HTML page as an alternate and is listed in llms.txt. The served content-type header should be checked, since a 200 status alone does not prove the file displays.

All methods