Learning

This is Unit Eight of Artificial Intelligence: learning. The unit runs long, so it comes in two parts, and this is the first. The approach here is the one that now dominates the field. Instead of writing rules, supply examples — inputs paired with correct outputs — and let a procedure build a predictor from them. The unit imposes one discipline before anything else happens, and it is the most important habit in the subject. Part of the data gets locked away untouched, and the learner never sees it. A predictor graded on what it was trained on is being tested on its memory. That distinction produces the unit's central number: the gap between how well a learner does on what it studied and how well it does on what it has never seen. Everything about overfitting is a way of talking about that gap. And the unit is honest about what learning did and did not solve. Unit Five's bottleneck was writing knowledge down. This approach does not remove that difficulty — it moves it. By the end you will be able to do five things. One: say why part of the data is locked away first. Two: say what a test score transfers to. Three: state overfitting, and why it is managed rather than solved. Four: say where learning moved the bottleneck. Five: say what more data fixes and what it cannot.

Three entries. What the approach is, the small case the whole unit runs on, and a discipline applied before any learning happens — which is the habit worth taking away even if nothing else survives.

The approach.

AI 0401 Learning from Examples

Training examples — inputs with their correct outputs — turned into a predictor for cases nobody has seen. The examples do the work that written rules did in the previous units.

The case.

AI 0402 The Stall Dataset

A tiny dataset carrying every lesson in the unit — eight days of a market stall, with weather, weekend, festival and result. Small enough to check by hand throughout.

The discipline.

AI 0403 The Test Drawer

Part of the data set aside untouched before learning begins, and not looked at until grading. A learner graded on what it studied is being tested on its memory rather than on anything it learned.

Four entries on two ways of building a predictor. One stores everything and copies from the most similar case. The other asks questions in sequence — and produces something you can read.

The first method.

AI 0404 Nearest Neighbor

Store the training examples and answer a new case by copying the most similar stored one. There is no training step at all — the examples are the model.

What it depends on.

AI 0405 The Tie in the Ruler

The method stands or falls on its measure of similarity. A coarse measure produces ties between neighbours with opposite answers, and then the prediction is arbitrary.

The second method.

AI 0406 The Decision Tree

Predicting by asking questions in sequence and branching on the answers, with each question chosen to split the remaining examples most usefully.

Grown by hand.

AI 0407 Growing the Stall Tree

A full run on six rows, with candidate questions auditioning and piles splitting — producing a readable theory of the task rather than an opaque one.

Three entries producing the unit's central number, and then an error entry about what that number licenses — which is less than people take it to license.

The rule.

AI 0408 Grading Day

A predictor is graded on examples it has never seen. The score on training data measures memory, and only the held-back score measures anything else.

The number.

AI 0409 Opening the Drawer

Grading on the locked rows produces the most important number in the unit — the gap between how well it did on what it studied and on what it had not seen.

And the error.

AI 0410 "It Scored Well, So It Understands"

A test score is agreement on cases drawn the same way the test cases were. It transfers as far as a new setting resembles the drawer it came from — and no further, whatever the number says.

Two entries on the failure mode this whole apparatus exists to detect. And the second is careful in a way worth noting — it calls the remedies management rather than a cure, because the problem does not go away.

The failure.

AI 0411 Overfitting

Extracting regularities that genuinely exist in the training examples and do not exist in the world. The pattern is real where it was found — that is what makes it hard to notice.

The remedies.

AI 0412 The Remedy Trio

Three standing responses: hedge the counts, charge for complexity, and hold data back. They manage the problem rather than abolishing it — a learner can always find structure in a finite pile.

Two entries closing Part One with the honest accounting. Learning did not remove Unit Five's difficulty about acquiring knowledge — it relocated it. And the obvious fix works on one kind of problem and not the other.

Where it moved.

AI 0413 The Pile Is the Bottleneck

A learner is exactly as good as its examples are representative. The bottleneck moved from writing knowledge down to collecting the right pile — which is a different problem, not a solved one.

And the error.

AI 0414 "More Data Always Helps"

More data fights accidents — flukes disappear at scale. Only representativeness fights skew, and no quantity of the same slanted material corrects the slant. The two problems need different fixes.

Five things you can now do — and Part Two builds the machinery that dominates the field. You can say why part of the data is locked away before anything starts, because a predictor graded on what it studied is being tested on its memory. You can say what a test score transfers to — cases drawn the way the test cases were, and no further, whatever the number happens to be. You can state overfitting and say why it is managed rather than abolished: a learner can always find structure in a finite pile. You can say where learning moved the bottleneck — from writing knowledge down to collecting a representative pile, which is a different problem rather than a solved one. And you can say what more data fixes and what it cannot: quantity kills flukes, and only representativeness touches skew.