Study guide · Neural networks · 1958

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

Frank Rosenblatt · Psychological Review

Rosenblatt asks how a brain could store what it sees and use it to recognise things later. His answer is a randomly wired network of simple cells that learns from experience by strengthening the cells that were active when it got the answer right: the first machine that learns to classify patterns from examples.

The paper in four sentences

Your memory of a face is not a stored photo. It is a change in connections, so that the same kind of input comes to produce the same reaction.

Rosenblatt's machine has three layers: light sensors, a crowd of randomly wired "association" cells, and a few answer cells that compete to respond.

Learning means making the association cells that were active during a correct answer count for more next time.

Shown random patterns, it can only memorise. Shown real categories (squares vs circles), it learns to recognise new examples too. What it cannot grasp are relations, like "the thing to the left of the square".

Idea · pp. 386–388

Memory is connections, not copies

How does the brain remember a face? One view says it keeps a copy, like a photo in a drawer, and compares each new face to the drawer. Rosenblatt takes the other view: nothing is stored as a picture. Instead, experience changes how strongly cells connect, and a familiar face simply flows down the strengthened paths to the right reaction. Recognising is the flowing; there is no separate look-up step.

He also has a practical point. Fifteen years earlier, McCulloch and Pitts described brains as exact logic circuits. Real brains are messy and partly random. So Rosenblatt builds his machine with random wiring on purpose, and studies it with probability instead of logic.

Everyday pictureA path across a lawn. Nobody draws a map of where people walk. The grass just wears down where people go most, and new walkers follow the worn path without thinking.

TakeawayIn a perceptron, memory lives in connection strengths, not in stored copies of what was seen.

Check yourself
Why did Rosenblatt use random wiring?
Because real brains are not wired exactly. He wanted to show learning could work even without a precise design, and he analysed it with probability.

Idea · pp. 389–392 · Figs. 1–3

Sensors, a random crowd, and competing answers

The machine looks at a small grid of light sensors, its "retina". Each sensor is on if light hits it. Behind the retina sits a crowd of association cells. Each one is wired to a handful of randomly chosen sensors, and it switches on if enough of its sensors are lit. Behind the crowd are two answer cells, say "Answer 1" and "Answer 2". Each answer cell listens to its own random group of association cells, and the louder group wins.

The random wiring never changes. What changes is how much each association cell's voice counts. Rosenblatt calls this its value. When you tell the machine "that was Answer 1", every association cell that was on and belongs to Answer 1's group gets a bit louder. Next time a similar picture lights up many of the same cells, Answer 1 wins.

Try it: draw a shape, train it, then test it on a shifted or slightly different shape.

Click squares on the retina to draw. Amber association cells are on. Each answer adds up the values of its own active cells (left column: Answer 1's group; right column: Answer 2's). Try: load "Vertical bar", reinforce Answer 1; load "Horizontal bar", reinforce Answer 2; repeat twice; then test the shifted bars. A small simulation in the spirit of the paper, not its exact parameters.

TakeawayRandom fixed wiring turns a picture into a pattern of active cells. Learning only adjusts how much each cell counts toward each answer.

Check yourself
Which part of the perceptron learns?
Only the "values" of the association cells, that is, how strongly they push on the answers. The sensor-to-association wiring stays random and fixed.
Why can it recognise a shifted bar it never saw?
A shifted bar still lights up many of the same association cells as the trained bar, because each cell samples scattered sensors. Shared active cells carry the learned values over.

Idea · pp. 396–402

Real categories make generalising possible

Rosenblatt runs two thought experiments. In the first, the pictures are pure noise, random dots, and you label half "1" and half "2" for no reason. The machine can memorise some of them, but the more it learns, the more it forgets, and on new noise it guesses at chance. There is nothing to generalise from.

In the second, pictures come in real groups, like squares and circles. Now the members of a group light up overlapping cells, and learning one square helps with the next. The machine gets new examples right as often as old ones, and adding more association cells pushes it toward 100%.

Everyday pictureMemorising a phone book teaches you nothing about the next number. Learning that dogs bark and cats meow lets you guess right about a dog you have never met.

TakeawayA learning machine can only generalise if the data has real structure: things in the same category must look more alike than things in different ones.

Check yourself
What happens on random, meaningless labels?
It memorises a bit, then gets worse as it learns more, and does no better than chance on new examples.

Idea · Later form (1962)

The learning rule as it is taught today

The 1958 paper describes learning in terms of cells gaining value. The tidy version every textbook now shows came from Rosenblatt's 1962 book: give each input a weight; add up weight × input; answer "yes" if the total is above zero. When the answer is wrong, nudge the weights toward the right answer. When it is right, leave them alone.

Geometrically, the weights draw a straight line through the data, and every mistake tilts the line. Press "Step" to watch it learn. Then switch to the "crossed" data, where no straight line can separate the two colours: the line keeps tilting forever. That limitation became the subject of a famous 1969 book.

Each step shows one point (circled). If it is on the wrong side, the weights move by η(target − guess)·input and the line tilts. On separable data the mistakes stop. On crossed data they never do.
Later refinementThis error-correcting rule, w ← w + η(y − ŷ)x, is the textbook form from Rosenblatt's Principles of Neurodynamics (1962), not the 1958 paper. The 1962 book also proved that it always finds a separating line if one exists (the perceptron convergence theorem).

TakeawayA perceptron draws one straight boundary and fixes it by nudging after each mistake. It succeeds when a straight boundary exists and fails forever when one doesn't.

Check yourself
When does the perceptron change its weights?
Only after a mistake. Correct answers leave the weights unchanged.
Why can't it learn the crossed data?
No single straight line separates the colours. Every fix for one point breaks another.

Idea · pp. 404–405

Where it stops: relations

Rosenblatt is honest about the limits. His perceptron can learn "say square when you see a square". It cannot learn "name the object to the left of the square", or "point to whatever came before the circle". Those answers depend on a relationship between things, not on which cells light up.

He concludes that recognising relations would need a system "more advanced in principle". That turned out to be right: it took networks with several learning layers, trained by backpropagation, to learn their own internal features.

TakeawayA single layer of learning handles concrete patterns. Relations in space and time need more.

Follow the idea

Built on: Artificial neuron (1943)

Led to: Perceptrons (book) (1969), Backpropagation (1986)