Human-in-the-Loop Patterns

This is Unit Six of Generative AI at Work, on keeping people in the loop. Unit Five checked outputs. This unit designs the human side of the system, so that checks and decisions actually happen. It draws on NIST's framework and Generative AI Profile, and on the M-CaReM-AI method. Carry one question through this unit: what makes a human in the loop real rather than a name on paper? The answer arrives in a validation session from the method's composite case. By the end you will be able to describe human–AI configurations and roles. To recognize automation bias and algorithm aversion. To read overrides, the rewrite ratio and incidents. To apply the feature gate and a three-level review rule. And to keep the loop honest with logs and reporting.

Four entries on where a task sits between fully automatic and fully manual, and on naming who does what.

A range, not a switch.

GAI 126 Autonomous to Manual

NIST's AI Risk Management Framework describes human–AI configurations as a range from fully autonomous to fully manual, with most work in between: a spam filter acting alone, a doctor writing every word of a note, and an AI draft a person edits. Where a task sits on the range is a decision someone makes, and it should be made on purpose.

Three roles an AI can take.

GAI 127 Three Roles for the AI

NIST's framework names three roles: deciding on its own, deferring the decision to a human expert, or giving a human decision-maker an additional opinion. One tool might route routine invoices itself, send unusual ones to a clerk, and suggest codes the clerk can change, so say which role applies when.

When no oversight is needed.

GAI 128 Some Systems Need No Oversight

NIST notes that some AI systems may not require human oversight, citing models that improve video compression, while others specifically require it. Automatic image sharpening in a video call runs unattended; a model drafting benefit decisions does not. The deciding question is what a wrong output can do, not how clever the system is.

Naming who does what.

GAI 129 Roles Must Be Named

NIST's framework says human roles in deciding and overseeing AI must be defined and differentiated, so the right people are empowered, responsible and trained, such as a one-page note naming who drafts, who checks, who approves and who handles complaints. When everyone assumes someone else checked, nobody did.

Five entries on trusting AI too much and too little, and on what a real human decision needs.

Trusting AI too much.

GAI 130 Automation Bias

Automation bias, in NIST's profile, is excessive deference to automated systems, including judging AI content as better than other sources without reason, like a reviewer approving an AI draft faster than a colleague's because the tool is usually right. NIST notes that automation bias makes other risks worse, especially confabulation and bias.

Trusting AI too little.

GAI 131 Algorithm Aversion

The opposite failure, which NIST also names, is algorithm aversion: experts avoiding AI tools unnecessarily and losing their real benefits, like an analyst who rejects every suggestion on principle and spends a week on a task the tool could have drafted in minutes. Good oversight is calibrated: neither rubber stamp nor reflexive veto.

What deciding requires.

GAI 132 Humans Decide, Again

The method's first principle, AI proposes and humans decide, only works if the deciding person has the authority, the time and the information to say no. A reviewer with five minutes for forty AI drafts is approving them, not deciding. A human in the loop on paper is not a human in the loop in practice.

Documents AI may not change.

GAI 133 Controlled Artifacts

The method treats approved documents and definitions as controlled: AI may propose changes, but cannot approve or deploy them without human review, so a suggested change to an approved definition arrives as a tracked proposal for its owner. The danger is the quiet edit, a small AI change to approved material that nobody reviewed.

Why generative AI gets extra review.

GAI 134 Extra Review for Generative AI

NIST's profile notes that generative AI is less well understood than other AI tools, so its use may warrant additional human review, tracking and documentation, and greater management oversight, such as a team adding a separate review step for anything a chat tool drafts. Existing AI controls may not cover generative tools, so check rather than assume.

Four entries on the signals a working loop produces: overrides, the rewrite ratio, feedback and incidents.

Whether people can really overrule it.

GAI 135 Can People Overrule It?

NIST's framework calls for more study of how far people are empowered and encouraged to challenge AI output, and suggests collecting data on how often and why they overrule it, such as a log showing reviewers changed one recommendation in three last month. Overruling should be easy and expected, not an exception that needs justifying.

Measuring the rewriting.

GAI 136 The Rewrite Ratio

The method suggests tracking an AI rewrite ratio: roughly how much of each AI draft people must rewrite before it is usable. One team rewrote most of each draft in month one and little by month three, after improving prompts and reference material. The ratio should fall over time; if it does not, the prompts or grounding need work.

Learning from users.

GAI 137 Feedback Loops

NIST's profile recommends gathering structured feedback from users, field testing and red-teaming, before and after a generative system goes into use, and feeding it into monitoring, such as a this answer was wrong button whose reports go to the prompt maintainers. Feedback that nobody reads trains people to stop giving it.

What counts as an AI incident.

GAI 138 AI Incidents

NIST describes an AI incident as an event where an AI system's development, use or malfunction contributes to harm to people, critical infrastructure, rights or legal obligations, or property, communities or the environment, such as an AI-drafted notice that gave customers a wrong deadline. NIST notes there are no formal reporting channels yet, so use your organization's process.

Four procedures for designing the loop: the feature gate, the boundary, review levels and escalation.

The method's gate for AI features.

GAI 139 The AI Feature Gate

The M-CaReM-AI method proposes a gate every AI feature must pass before it is built, and calls it the one step to take if a team does nothing else for AI governance. Name what the AI reads. Name what it writes or influences. Name how its outputs are reviewed, by whom, and before what. Then draw the boundary between what it suggests and what the system decides. The gate is the method's judgment, not a regulation; its value is forcing the answers before the feature exists, while changing them is cheap.

Suggests or decides.

GAI 140 "AI Suggests" or "System Decides"?

The gate's boundary separates suggestions for a person from outputs the system acts on. List every output the AI produces. For each, ask what happens if nobody looks at it. Anything that would change a decision, record, status or customer message without a person goes on the decides side. Move all of those behind human approval, unless an error is minor and automation is agreed in writing. Boundaries drift: a suggestion that becomes a rarely changed default has quietly crossed over.

Matching review to the stakes.

GAI 141 A Three-Level Review Rule

This course's three-level rule is built from NIST's framework and the method. Rate the consequence of an error: low, moderate or high, where high touches safety, money, legal obligations or customer trust. Low gets a light check by the user. Moderate gets a full claim-by-claim check, with expert review where specialist. High is a draft for a named person who decides; the AI never decides. NIST supports this: human intervention is needed where a system cannot catch its own errors. Decide each task's level in advance, before time pressure lowers it.

When to escalate.

GAI 142 When to Escalate

Escalation needs triggers set in advance. Escalate when a check finds an error you cannot resolve from the source. Escalate when output touches a high-consequence area and you are not the named decision-maker. Escalate when sources disagree in a way that affects the outcome. Escalate when the output conflicts with your professional judgment, and say so. Then record the escalation and its result. The fourth trigger is the one people skip, because overruling a confident tool feels presumptuous.

Three procedures that keep the loop honest, then two cases. The opening question is answered here.

The AI-use log.

GAI 143 Keeping an AI-Use Log

An AI-use log records where AI drafted or revised work, which the method requires for important documents and NIST's profile supports. Decide what gets logged: at least high-consequence work, anything sent outside the team, and changes to controlled documents. Record the date, tool, task and output. Record the check and who did it. Record corrections and overrides with reasons. Review it monthly. Start with the minimum, because a log that is too detailed will not be kept.

Tracking overrides.

GAI 144 Tracking Overrides

NIST suggests collecting data on how often and why people overrule AI output, and the rewrite ratio measures the same thing from the other side. Mark each output accepted, edited or rejected. Record a short reason for edits and rejections. Count outcomes and reasons monthly. Then act: fix prompts for tone, grounding for facts, and reviewer training for wrong rules that got through. A team that never overrides deserves a closer look, in this course's reading: the tool may be excellent, or reviewers may not be reviewing.

Reporting an incident.

GAI 145 Reporting an Incident at Work

Because there are no formal external channels yet, workplace reporting runs through your organization's incident process. Stop the harm first: correct the output, pause the use or warn the people affected. Report through the normal route, saying an AI system was involved and how. Preserve the prompt, output, sources and log entry. Feed the lesson back into prompts, review levels or the gate. AI involvement is not an excuse; the same accountability applies.

A loop designed for one task.

GAI 146 Designing the Loop for One Task

A worked example, constructed for this course, follows an insurance claims team using AI to draft letters explaining claim decisions. The gate says the AI reads claim records, decision fields and templates, never medical notes, and writes only drafts. A wrong letter about money would reach a customer, so letters stay suggestions, reviewed at the high level: the handler checks every amount, date and reason and approves by name, and denials get a second reviewer. The log shows the rewrite ratio falling, and the second denial review is kept anyway, because a falling error rate is when automation bias sets in.

The person who knew the work.

GAI 147 The Reality Check in the Room

The method's composite hospital case includes a validation session showing why AI can help the room but must never be the room. AI had prepared a one-page summary, scenarios and a list of disagreements. Then a frontline nurse asked whether an incident could be closed clinically while staying open administratively. The AI material treated closure as one event, as generic documentation does, and the team changed the design. AI is the scribe and summarizer; it must not validate on generic patterns or resolve policy disagreements. That answers this unit's question: a human in the loop is real when the people who do the work are in the room, not only signing at the end.

Look at what you can now do. You can place a task on the range from autonomous to manual, and name the AI's role. You can recognize automation bias and algorithm aversion, and say what a real decision needs: authority, time and information. You can read overrides, the rewrite ratio, feedback and incidents as signals. You can apply the feature gate, the suggests-or-decides boundary and the three-level review rule. And you can keep the loop honest with a log, override tracking and incident reporting. Unit Seven turns to privacy, vendors and governance.