Skip to main content

One Row per Word

This video presents the same text shown beside it, spoken and on screen. It adds nothing the text does not say.

State

A fixed embedding gives each word one row forever — "bank" occupies a single position whether the sentence means rivers or loans.

Show

"She fished from the bank" and "she robbed the bank" hand the model identical rows for their pivotal word.

Watch for

Fixing the flaw requires context to bend the rows at reading time — the transformer's entrance.