Generative AI at Work / Unit 1 GAI 017 worked example, 60 to 90 seconds
Reading an AI Answer
The idea
Reading a generated answer well means looking past its fluency to four things that show whether it can be relied on.
In practice
Step one: find the claims. List the facts, numbers, names and citations the answer depends on. Step two: trace each one to a source. Anything drawn from material you supplied can be checked against it; anything from the model's general knowledge needs an outside check. Step three: look for inconsistency, an answer that contradicts your input or its own earlier statements, which NIST also counts as confabulation. Step four: test the reasoning. Steps that look logical can still lead to a wrong conclusion, so check the conclusion directly.
Where people slip
The M-CaReM-AI method names the failures to watch for: invented fields, rules that are confident but wrong, and definitions that drift over time.
Your place on the trail
Every lesson rests on earlier ones and opens later ones. Tap any to jump.
Something wrong in this lesson? Flag it · Sources for Generative AI at Work