Skip to main content

The Bigram Table

This video presents the same text shown beside it, spoken and on screen. It adds nothing the text does not say.

State

Counting how often each word follows each word in a pile of text yields a working language model — the bigram model, built from shares alone.

Show

The corpus, four sentences whole: the cat sat; the cat purred; the dog sat; the dog ran. Count the pairs. After "the": cat twice, dog twice — shares one half and one half. After "cat": sat one half, purred one half. After "dog": sat one half, ran one half. That table is a language model. And it already generates: start at "the," sample from its column, move to the sampled word, repeat. "The cat purred." "The dog sat." Fluent, at toy scale, by counting alone — no grammar written anywhere, no meanings defined, arithmetic on a four-sentence world.

Watch for

Every share is a Unit 6 count wearing a new job title — the model is only as wide as its table.