Learning

This is Unit Eight of Artificial Intelligence, Part Two: weights, layers, and descent. Part One covered learning in general. This part builds the specific machinery that dominates the field now — and it builds it small enough to train by hand, from all-zero settings, with every step checkable. The sequence is worth watching because it is a sequence of limits and escapes. The simplest machine has a hard limit, proved in four lines. Stacking copies of it buys nothing at all unless one ingredient is added. And the fix that makes deep training possible costs the property the course valued most in Unit Five. That last point is the one to carry. A trained simple machine can be read — its settings state a theory of the task you can argue with. A large trained network cannot, and verification stops being reading and becomes something else entirely. By the end you will be able to do five things. One: say what the simplest machine is, and what its knobs amount to. Two: say why the brain comparison misleads. Three: say why stacking buys nothing without a bend. Four: say what descent finds, and what it does not. Five: say what gets lost as networks grow.

Four entries turning a described case into something arithmetic can act on. A row of measurements, a score expressing a point of view about them, a verdict — and then what that whole arrangement amounts to geometrically, which is one straight cut.

The representation.

AI 0421 The Row

Once features are measured rather than named, a case becomes a row of numbers — one per feature, in a fixed order. Everything after this operates on rows.

The score.

AI 0422 The Weighted Sum

Multiply each number in the row by a matching weight and add. The result is one score, and the weights are a point of view about which features matter.

The verdict.

AI 0423 The Threshold and the Balance

A score becomes a decision by comparison against a threshold — and folding that threshold into the sum as a starting balance makes it one more weight.

What it amounts to.

AI 0424 The Boundary

Every weighted sum with a threshold divides the space of possible cases into two sides separated by a straight boundary. That geometry is the machine's entire capability.

Three entries. The machine itself, an analogy the course handles carefully, and the rule by which it learns — which is simple enough to run in your head.

The machine.

AI 0431 The Perceptron

A weighted sum with a starting balance and a yes-above-zero verdict. The knobs are all there is to it — there is no other structure hiding anywhere.

The analogy.

AI 0432 "Neural Networks Work Like the Brain"

These are brain-inspired the way a paper airplane is bird-inspired. What actually runs is multiply, add, and compare — and the inspiration explains the naming rather than the mechanism.

How it learns.

AI 0433 The Nudging Rule

A correct verdict changes nothing, and every error shifts each weight one step toward getting that case right. Learning here is entirely error-driven.

One capstone, and its value is that nothing is hidden. Starting from all zeros, four passes produce a working machine, and every step is arithmetic you can perform yourself.

The capstone. From all-zero knobs.

AI 0434 The Training Run

Trained by hand from settings of zero, the machine learns to sort in four passes over the examples. Every step is checkable arithmetic — nothing about it requires taking anything on trust.

Two entries. What a trained machine of this kind is — which is something you can read and disagree with — and a convergence promise with an important condition attached to it.

What it is, once trained.

AI 0435 Reading the Knobs

The final weights state a theory of the task — which features mattered and how much. That theory can be audited, argued with, and rejected, which is a property worth noticing before it disappears.

The guarantee.

AI 0436 Convergence, If Separable

Whenever some straight boundary can separate the two groups, the nudging procedure is guaranteed to settle. When none exists, the guarantee says nothing — and the next topic is about that case.

Two entries, and the first is a genuine impossibility proof in four lines of arithmetic. The second draws the general lesson, which applies well beyond this particular machine.

The proof.

AI 0437 The Either-Or Wall

Four lines of arithmetic show that no machine of this kind can sort an either-or-but-not-both pattern — not by training, not by hand-setting, and not by any amount of inspiration.

The general lesson.

AI 0438 "A Learning Machine Gets There Eventually"

Training is search inside a space of possible settings, and no amount of searching finds a setting the space does not contain. A learning machine does not get there eventually if there is no there.

Two entries on the way out, and the second is the surprise. Stacking these machines is the obvious move, and it accomplishes exactly nothing unless one specific ingredient is added between the stages.

The move.

AI 0439 Stack the Cuts

Several units read the case and another reads their verdicts — two stages, drawing several cuts instead of one. That is the escape from the single-boundary limit.

And what it needs.

AI 0440 Bends Required

Stacking straight stages buys nothing — a weighted sum of weighted sums is another weighted sum, and the whole stack flattens back to one cut. Each stage needs a bend in it for depth to mean anything.

Two entries. What the word actually means, which is less mysterious than it sounds, and why the hard yes-or-no threshold had to be replaced before deep training could work.

What depth is.

AI 0451 Depth Is Composition

A stack of layers, each re-describing the case for the next one. Deep means nothing more than how many times that happens — the word is a count.

Why the threshold went.

AI 0452 Gentle Bends

A flat yes-or-no verdict starves deep training, because a knob buried in the stack can move without flipping any verdict, so nothing registers. Smooth bends replace it so small changes have small effects.

Four entries on how a smooth machine gets trained. A landscape image, a procedure requiring no calculus, the one number controlling it, and an error entry about what the procedure actually achieves.

The image.

AI 0453 The Error Landscape

The knob settings are a position and the total error is the altitude. Training becomes walking on that landscape, looking for lower ground.

The procedure.

AI 0454 Downhill by Feel

Nudge a knob, measure how the error changed, step in the direction that lowered it, repeat. It can be performed without any calculus at all — the calculus is an accelerant.

The stride.

AI 0455 The Learning Rate

How far each step moves is the learning rate, with each weight moving by the stride times its own influence. That is the anchor rule of modern training.

And the error.

AI 0456 "Training Finds the Best Possible"

Descent finds a valley rather than the deepest valley — large terrain holds canyons and saddles. The truthful claim is that it found somewhere low, not that it found the bottom.

Three entries. How responsibility gets distributed through a stack, what capability actually scales with, and an error entry about scaling that echoes the one from Part One.

Distributing blame.

AI 0457 Backward Blame

Every knob in the stack gets its share of the blame, passed layer by layer from the output back toward the input in a single sweep.

What scales it.

AI 0458 Scale: The Three Dials

Once layers, bends, descent and blame exist, capability scales with quantities you can buy — examples and computation. That purchasability is why the field went the way it did.

And the error.

AI 0459 "Bigger Is Always Better"

A bigger model without more data feeds the appetite for memorising. More data of the same slant polishes the slant. This is Part One's lesson about quantity and representativeness, at scale.

One entry closing the unit, and it is the one this whole part has been building toward. The readable theory from Topic Four does not survive the journey to large networks.

What is lost.

AI 0460 The Last Readable Knobs

A trained deep network's theory of its task is written across millions of coupled settings and cannot be read. Verification stops being reading and becomes something else — which is where Unit Five's audit trail finally ends.

Five things you can now do, and Unit Eight is complete. You can say what the simplest learning machine is — a weighted sum with a starting balance and a verdict — and that its knobs are the whole of it. You can say why the brain comparison misleads: what runs is multiply, add and compare, and the inspiration explains the name rather than the mechanism. You can say why stacking buys nothing without a bend — a weighted sum of weighted sums flattens back into one cut. You can say what descent finds: a valley rather than the deepest valley, so the honest claim is that it reached somewhere low. And you can say what is lost at scale — a readable theory becomes millions of coupled settings, and verification stops being reading.