Language, Vision, Bodies

This is Unit Nine of Artificial Intelligence, Part Two: vision and bodies. Both subjects here get the same treatment, and it is a deflating one in a useful way. A vision system receives a grid of numbers and produces scores. Nothing sits between those two things — no scene, no gaze, nothing that could be called looking. The technical content is Unit Eight's limit met again. A single straight cut through the space of images cannot recognise an object, because the same object in a different position produces completely different numbers. The answer is to slide a small template across the grid instead. Then bodies, where everything gets harder at once. Unit One described a table of environment properties, and an embodied agent occupies every difficult cell of it simultaneously — partially observable, uncertain in its effects, and running in real time. And the part closes on two lessons that transfer well beyond robotics: why the order in which machines conquered tasks was inverted, and what a demonstration does and does not establish. By the end you will be able to do five things. One: say what a vision system receives and produces. Two: say why one cut fails here, and what replaces it. Three: say why control runs at several speeds at once. Four: say why physical experience is expensive. Five: say what a demonstration licenses.

Two entries, and the second is the deflation. What arrives is a grid of brightness values. What leaves is a set of scores. People imagine a great deal in between, and there is nothing there.

What arrives.

AI 0531 A Grid of Numbers

An image reaches a machine as a grid of numbers, one brightness value per cell, and nothing else. Everything a vision system does operates on that grid.

And what does not happen.

AI 0532 "The Computer Sees the Picture"

The system takes a grid and produces scores. There is no scene assembled in between, nothing that gazes, and nothing that could be called looking — the word see is doing work the machinery does not do.

Three entries. Unit Eight's single-boundary limit reappears in a form specific to images, and the fix is a small move that turns out to have a large consequence.

The failure.

AI 0533 Why One Cut Fails

One tilted cut through the space of images cannot recognise objects, because the same object in a corner and in the centre produces entirely different numbers. Position destroys the resemblance the cut needed.

The move.

AI 0534 The Sliding Template

Slide a small template of weights across the grid, computing the same weighted sum at every position. What was one fixed cut becomes one pattern, checked everywhere.

What comes out.

AI 0535 The Edge Map

Sliding a square template across the grid produces a map — one match score per position. The output is not a verdict but a picture of where the pattern was found.

Four entries. Where the templates come from, the property making them efficient, the stack built out of them, and an entry refusing to exempt any of it from Unit Eight's costs.

Where they come from.

AI 0536 Templates Are Learned

The templates are not designed — their weights are settings like all the others, sculpted by descent against training examples.

The efficiency.

AI 0537 Weight Sharing

The same template weights are used at every position, so a pattern learned in one part of the image is recognised in all of them. One set of settings covers the whole grid.

The stack.

AI 0538 The Seeing Stack

Template maps feeding further template maps, with bends and coarsening between the stages. It is Unit Eight's depth, arranged for grids.

And no exemption.

AI 0539 Seeing Is Not Exempt

A seeing stack's competence is its training distribution's competence, and its deeper templates are as unreadable as any other deep settings. Nothing about vision escapes the costs from Unit Eight.

Four entries on what changes when an agent has a body. Nothing arrives labelled, so perception has to build the usable view from raw physics — and the control structure that follows runs at several speeds simultaneously.

What changes.

AI 0546 The Unpackaged World

An embodied agent gets no pre-packaged stimuli. Perception has to construct the labelled view out of raw physical signal, which earlier units quietly assumed was already done.

The structure.

AI 0547 Layers of Control

Control runs in layers at different speeds — fast lower layers reacting within fractions of a second, slower upper layers deliberating. They operate simultaneously rather than taking turns.

What each layer does.

AI 0548 Remember, Command, Tell

Each layer keeps its own belief, issues its own commands, and reports upward — updating on its own clock rather than on a shared one.

Worked.

AI 0549 The Floor Cleaner, Layered

One floor-cleaning robot built in three layers of control, each running at its own rate. Seeing the whole hierarchy at once is what makes the arrangement make sense.

Three entries locating robotics on Unit One's table. It occupies every difficult cell at once — and then the specific cost that follows, which is that experience here cannot be had cheaply.

Where it sits.

AI 0550 Every Hard Cell at Once

Robotics occupies every difficult cell of the environment table simultaneously — partial visibility, uncertain effects, real time, and more. Nothing is being held constant.

The cost.

AI 0551 Experience Is Expensive

The physical world does not label itself. A fall teaches at the price of a fall, and whether a grasp succeeded has to be established rather than read off. Examples cost real resources.

The response.

AI 0552 Borrowed Experience

Every answer the field has borrows experience from somewhere cheaper — learning in simulation and then crossing the gap to reality being the main one.

Four entries closing the unit, and the first two transfer well past robotics. Why machines mastered chess before walking, and what a demonstration actually establishes about a system.

The inversion.

AI 0553 The Inverted Ladder

Machines conquered chess long before walking, because how difficult a task feels to a human never measured how difficult it is in general. Our sense of difficulty tracks our own unusual competences.

What a demonstration is.

AI 0554 A Demo Is a Chosen World

A demonstration is a system meeting an environment, and that environment was chosen — the lighting, the floor, the objects, the sequence. The choice is part of the result being shown.

And what it licenses.

AI 0555 "The Demo Robot Is the Product Robot"

A flawless demonstration licenses no conclusion about an uncontrolled setting. Every difference between the chosen environment and a real one is a place where the system can fail.

The stakes.

AI 0556 Stakes with Contact

Learning transformed perception and action at the familiar prices — appetite for data, fragility outside the training distribution, opacity — now attached to systems that touch the world.

Five things you can now do, and Unit Nine is complete. You can say what a vision system receives and produces — a grid of numbers in, scores out — with nothing between them that could be called looking. You can say why one straight cut fails on images, since position changes the numbers entirely, and that sliding a template everywhere is the answer. You can say why an embodied agent runs control at several speeds at once, with each layer keeping its own belief on its own clock. You can say why physical experience is expensive — a fall teaches at the price of a fall — and that the field's answers all borrow experience from somewhere. And you can say what a demonstration licenses: nothing about an uncontrolled setting, because the environment shown was chosen.