Generative AI at Work / Unit 6 GAI 144 worked example, 60 to 90 seconds
Tracking Overrides
The idea
NIST suggests that data on how often and why people overrule AI output is worth collecting, and the method's rewrite ratio measures the same thing from the other side.
In practice
Step one: in the AI-use log, mark each output as accepted, edited or rejected. Step two: for edits and rejections, record a short reason: wrong fact, wrong rule, wrong tone, or missing information. Step three: each month, count the outcomes and the reasons. Step four: act on the pattern, by fixing prompts for repeated wrong tone, grounding for repeated wrong facts, and reviewer training for repeated wrong rules that got through.
Where people slip
A team that never overrides deserves a closer look, in this course's reading of NIST's point. It may mean the tool is excellent, or it may mean reviewers are not really reviewing. Only a sample of re-checked outputs can tell which.
Your place on the trail
Every lesson rests on earlier ones and opens later ones. Tap any to jump.
Something wrong in this lesson? Flag it · Sources for Generative AI at Work