A Pre-Filled Field, Governed

In practice

A benefits agency proposes a generative AI feature that pre-fills the reason field on each Eligibility Determination for the caseworker. The AI Safety Lead, Sam, starts with Govern one point six: she checks the Determination's matrix and finds one AI system already listed, a risk-scoring model, and adds the new feature to the inventory. For Govern three point two, she applies the boundary test: if nobody edits the pre-fill, it becomes the reason sent to the family, so the feature sits on the system-decides side unless a person approves it. The team decides every reason must be read and confirmed by a caseworker. For Manage one point one, the purpose is written: consistent, plain reasons that cite the rule applied; it is not for choosing the outcome. Under Manage one point two and one point three, the main risk, a reason that sounds right but cites the wrong rule, is mitigated by a check against the rule-set version on the Determination. The remaining risk, a hurried caseworker approving a flawed reason, is written down as residual risk under Manage one point four. For Manage two point four, Sam names the program manager as the person who can switch the feature off, and the fallback is the old free-text field. Under Manage four point one, appeals that dispute a reason are routed back to the team, and the override log is reviewed monthly. Because the feature is generative, Sam doubles the review sample for the first quarter.