Verification and Fact-Checking

This is Unit Five of Generative AI at Work, on verification and fact-checking. Unit Four made prompts safer. This unit is about what happens after the output arrives: checking it. It draws on NIST's Generative AI Profile, NIST's framework, the M-CaReM-AI method and the DAIR.AI guide. Carry one question through this unit: when a polished answer arrives, how do you find the error a careful reader would miss? The answer arrives with a composite case from the method, where two errors were caught. By the end you will be able to explain why verification is needed. To use NIST's marks of trustworthy information. To use grounding, asking twice and provenance as aids. To check claim by claim. And to bring in experts and record what you checked.

Three entries on why verification is part of the job, and where errors hide.

Why checking never goes away.

GAI 101 Why Verify

NIST describes confabulation as a natural result of how generative models work, so better prompts reduce it but no prompt eliminates it. A carefully grounded prompt can still return one wrong date among twenty correct facts. Verification is part of the job whenever generated content will be relied on, not a sign that the prompt failed.

Why summaries need it most.

GAI 102 A Wrong Summary Can Harm

Summaries are where generated errors do the most quiet damage, because the people who read a summary are usually the people who will not read the source, such as leadership approving a renewal from a summary that misstated the notice period. NIST's confabulated patient summary is the same failure in its most serious form.

Citations that do not exist.

GAI 103 Invented Citations

NIST notes that generated answers can include confabulated citations: references that look real, with plausible authors, titles and section numbers, but do not exist or do not say what is claimed, such as a memo citing a regulation section about something else entirely. A citation is a claim like any other, so open it before relying on it.

Five entries on what trustworthy information looks like, and on testing with your own material.

A checklist from NIST.

GAI 104 Six Marks of Trustworthy Information

NIST's profile, drawing on a twenty twenty-two White House research roadmap, describes high-integrity information as accurate and reliable, separating fact from opinion and inference, acknowledging uncertainty, transparent about its vetting, linked to sources, and verifiable. These six marks make a ready checklist for any AI output, though an output can match their style while failing their substance.

Three kinds of statement.

GAI 105 Fact, Opinion, Inference

High-integrity information keeps facts that can be checked, opinions that express a view, and inferences drawn from facts apart. A generated briefing might call a vendor unreliable, an inference from two late deliveries, in the same tone as the delivery dates. Generated text blends the three smoothly, so separating them is the reader's job.

Tracing claims to sources.

GAI 106 Back to the Source

The fifth mark of high-integrity information is that it can be linked to its original sources with appropriate evidence, like each figure in a report followed by the document and page it came from. A claim with no traceable source is a claim you are vouching for yourself.

Why test scores are not enough.

GAI 107 Benchmarks Are Not Your Task

NIST cautions that a model passing tests designed for humans, such as professional licensing exams, is not thereby valid or reliable for that work. A tool can score well on a published legal exam yet misread your own contract templates, and NIST adds that sensitivity to wording and varied real use widen that gap.

Testing on your own material.

GAI 108 Test on Your Own Cases

NIST's framework says accuracy should be measured on realistic test sets that represent actual conditions of use, such as twenty of last month's real requests with known answers, tried before a team adopts a tool. The vendor's demonstration shows what the tool can do, not what it does with your material.

Five entries on what makes checking easier: grounding, asking twice, grounded tools, provenance and the method's defences.

How grounding helps the checker.

GAI 109 Grounding Makes Checking Easier

Grounding an answer in material you supplied does more than reduce errors: it makes the remaining errors checkable, because every claim has a known place to look. A grounded policy answer can be checked in minutes against the pasted policy. Ungrounded answers draw on the model's general training, which you cannot open and inspect.

Asking more than once.

GAI 110 Ask Twice

The DAIR.AI guide describes self-consistency: generate several answers and favour the one they agree on. The working version is to ask more than once and treat disagreement as a flag, as when the same question returns two different deadlines. Agreement is weaker evidence than a source, because two answers can share the same error.

Tools that search your documents.

GAI 111 Grounded Tools

Some tools retrieve passages from a document collection and add them to the prompt before answering, which the guide calls retrieval-augmented generation, such as an assistant that finds the relevant pages in a policy library first. The research behind it names traceability and currency as open problems, so check retrieved citations too.

Where content came from.

GAI 112 Provenance

NIST describes provenance tracking: recording where content came from, who or what created it, when, and how it changed, using metadata, watermarks or fingerprints, like a report history showing which sections AI drafted and who edited them. NIST notes that provenance records have limits of their own, which should be documented.

The method's three defences.

GAI 113 The Method's Failure Modes

The M-CaReM-AI method names three defences for AI-drafted work: cross-check against real policies and data, involve domain experts, and treat approved documents as controlled. The failures they answer are invented fields, rules that are confident but wrong, and definitions that drift. When the stakes are high, none of the defences is optional.

Six procedures for checking generated work, from the claim-by-claim base to resolving disagreement.

The base procedure.

GAI 114 Checking Claim by Claim

Claim-by-claim checking turns the marks of high-integrity information into a procedure. List every claim, one per line. Label each as fact, opinion or inference. Find each fact's source, in your material or one you trust. Mark each confirmed, corrected or unsupported, and remove or rewrite the unsupported. Check that each inference rests on confirmed facts and follows from them. Skipping the listing step is how the claim that matters gets missed.

Numbers get their own pass.

GAI 115 Checking Numbers

Numbers deserve a separate pass, because a model produces a plausible figure as fluently as a correct one. Find every number, including dates, amounts, percentages and counts. Trace each to its source and match it exactly, units included. Recompute anything the model calculated yourself. Check each number is attached to the right thing. A total that does not match its parts is the quickest signal that something upstream is wrong.

Names and quotations.

GAI 116 Checking Names and Quotes

Names, titles and quotations are where invented detail looks most authoritative. Confirm every person, organization, document and product exists and is named correctly. Check titles against a current source. Compare every quotation word for word with the original. Then read the passage around each quote to confirm its meaning in context. A real quote on the wrong person or out of context can do more damage than an invented one.

Checking against your policy.

GAI 117 Checking Against Policy

The method's first defence is cross-checking against the real policies the output is meant to reflect. Identify which policy each statement depends on. Open the current approved version and confirm its date. Compare each rule with the policy's wording, conditions and exceptions included. Flag anything that sounds like general best practice and confirm it with the policy owner. A rule true almost everywhere can still be wrong for your organization.

Checking for what is missing.

GAI 118 Checking for What Is Missing

Omissions are the hardest errors to see, because the output reads as complete. Before reading it, list what a complete answer must contain. Tick each item off as you find it. For anything missing, check whether the source covers it, and add it. Then resolve the model's open questions with a person who knows. Writing the checklist first matters, because the output's own structure shapes what you expect to see.

When two answers disagree.

GAI 119 When Answers Disagree

When two generated answers disagree, resolve it in order. Identify exactly where they differ and what they share. Go to the source for each disputed point instead of asking the model to pick. Correct the output if the source settles it. If not, treat the question as open and take it to a person with authority, then record the answer. Asking the model which is right adds a third opinion, not evidence.

Two procedures on people and records, then a composite case where both errors were caught. The opening question is answered here.

When to bring in an expert.

GAI 120 Expert Review

The method's second defence is involving domain experts, whom NIST's framework also lists among the people who should inform AI work. Decide which outputs need one by stakes and domain: clinical, legal, financial or safety. Give the expert the output and sources with unconfirmed claims marked. Ask them to check meaning and practice, not spelling. Record what they changed. Do the mechanical checks first, so expert time goes on judgment.

Keeping a record.

GAI 121 Recording What You Checked

The method asks teams to record where AI drafted or revised important documents, and NIST's provenance guidance points the same way. Note that AI was used, which tool, and for which parts. Note what was checked, against what, and by whom. Note what was corrected and why. Keep the note with the document. A record of checks that were not done is worse than none, so write down only what happened.

Two errors, caught in a composite case.

GAI 122 A Summary, Verified

The M-CaReM-AI method teaches with a composite, illustrative hospital case. In one scene, an AI-drafted summary of the incident record includes an invented risk-score field; the analyst lists fields against the policy and deletes it. In another, AI drafts a clean rule: all severe incidents are reportable. Validation with the hospital's quality and risk staff shows the real rule depends on incident type, harm level and policy version. Neither error was caught by reading for quality. Both were caught by checking against the real source and by expert review. That answers this unit's question: the error a careful reader misses is found by checking claims against sources, not by judging how they read.

Look at what you can now do. You can explain why verification never goes away, and why summaries and citations need it most. You can use NIST's six marks of trustworthy information as a checklist, and test tools on your own cases. You can use grounding, asking twice, grounded tools and provenance to make checking easier. You can check claim by claim, including numbers, names, policy and what is missing. And you can bring in experts and record what you checked. Unit Six turns to keeping people in the loop.