Artificial Intelligence / Unit 10 AI 0607 short lesson, about 20 seconds
Preference Tuning Relocates the Question
Captions available
The idea
Steering models by human judgments relocates rather than dissolves the specification problem — judgments are data, and inherit every question data invites.
In practice
Whose preferences, gathered how, representative of whom, gameable how — the pile questions, pointed at the steering wheel.
Where people slip
The frontier beyond — oversight of systems more capable than their overseers — is live and unresolved; equipped readers read it, not settle it.
Your place on the trail
Every lesson rests on earlier ones and opens later ones. Tap any to jump.
Unlocks
Something wrong in this lesson? Flag it · Sources for Artificial Intelligence