The trouble with checking the answer afterwards
The standard way to make AI output trustworthy is to let a model write, then check what it wrote. The arrangement helps. It also has a floor, and the floor is not a tuning problem.
What everyone is doing
Enterprise guidance in 2026 converged on a recognizable arrangement. Retrieve relevant passages. Let a language model write an answer from them. Then run a separate verification pass, often described as span-level checking, in which a second model reads each claim in the answer and decides whether the retrieved passages support it. Unsupported claims get flagged, removed, or sent back for another attempt.1
Variants abound. Some teams swap the second model for a similarity matcher. Some route through a knowledge graph so the facts have structure. Some run several models and take the consensus. The shape is constant: free generation, then inspection.
It is a sensible engineering response and it genuinely reduces fabrication. Nothing below argues that it does not.
Four reasons it has a floor
The checker is the same kind of thing it is checking
A verification model is a language model making a judgment about whether a passage supports a claim. That judgment has an error rate, and it is not independent of the error the generator made. Both models were trained on overlapping data with overlapping notions of what sounds right. When the generator produces a plausible fabrication, the checker is more likely than chance to find it plausible too. Two correlated judgments do not multiply into a strong guarantee.
The failures that get through are the dangerous ones
A checker catches the obvious cases. What survives is the confident, well-formed, topically perfect fabrication — precisely the output a human reviewer will also wave through. Filtering does not simply reduce the error rate; it changes the composition of what remains, concentrating it in the failures hardest to spot downstream.
Support is not a binary property
“Does this passage support this claim?” is often a genuine judgment call. A passage can support a weaker version of a claim, support it under an assumption, or support it in a way that a specialist would qualify. Real professional documents are full of exactly these gradations, which is why the work is done by professionals. A binary supported-or-not verdict flattens them, and the flattening is where the liability sits.
The guarantee degrades exactly where you need it
Checking performance is measured on evaluation sets. Real use drifts away from them: an unusual question, an unusually long document, a domain nobody anticipated, a badly scanned annex. The generator's error rate rises in these conditions and so does the checker's, at the same time and for the same reasons. A guarantee that weakens under unusual conditions is weakest at the moment somebody is relying on it.
The economics reinforce it
Every layer added to catch the previous layer's errors costs latency, money and operational complexity. Each has its own thresholds to tune, its own regressions when a model is upgraded, and its own monitoring. Teams end up maintaining a pipeline whose purpose is to compensate for a component they cannot constrain. This is what practitioners mean when they call these systems finicky: not that they fail often, but that keeping them working is continuous manual labor with no end state.
The alternative framing
A filter over a system that can fabricate is a probabilistic control. A system that cannot produce material outside its sources is a structural one. The difference shows up in the sentence you can put in front of a regulator, an auditor or a judge. “We check the output and catch most problems” invites a question about the rest. “The output is drawn from the source documents, and here is where each part came from” does not.
That is the property CiteOnly is built around. What it costs, and what it consequently cannot do, is set out on that page.
Related
What a fabricated citation costs sets out what happened when review processes at capable organizations did not catch it. What regulators will ask you to prove covers the obligations arriving over the next two years.