Parameter Space Inspector

the neural net, live

Watch a letter go through it

Each dot is a neuron: brighter fires harder, hollow is negative. Lines are the strongest signals flowing right now. Slide to pick a letter.

ask it

Give it a start; it carries on

what it thinks comes next, after your start

attention

Which earlier letters each letter looks at

Rows: each letter as it is read. Columns: the letters before it. Brighter: more attention. Every model you have used works this way, with thousands of these heads instead of a handful.

its alphabet

letter map

Each letter’s learned vector, flattened to two dimensions. Letters it treats alike end up close; nobody told it which letters are vowels.