Artificial Intelligence / Unit 9 AI 0513 short lesson, about 20 seconds
Pretraining
Captions available
The idea
A large language model is a transformer pretrained on next-word prediction over vast text collections — billions of knobs descending against prediction error.
In practice
More training text than a person reads in ten thousand lifetimes; grammar, style, idiom, and fact-shaped regularity fall out of one objective.
Where people slip
"Fact-shaped" is the honest term — descent rewards producing the text that typically follows, which overlaps truth imperfectly.
Your place on the trail
Every lesson rests on earlier ones and opens later ones. Tap any to jump.
Something wrong in this lesson? Flag it · Sources for Artificial Intelligence