The paper in four sentences
Your memory of a face is not a stored photo. It is a change in connections, so that the same kind of input comes to produce the same reaction.
Rosenblatt's machine has three layers: light sensors, a crowd of randomly wired "association" cells, and a few answer cells that compete to respond.
Learning means making the association cells that were active during a correct answer count for more next time.
Shown random patterns, it can only memorise. Shown real categories (squares vs circles), it learns to recognise new examples too. What it cannot grasp are relations, like "the thing to the left of the square".
Idea · pp. 386–388
Memory is connections, not copies
How does the brain remember a face? One view says it keeps a copy, like a photo in a drawer, and compares each new face to the drawer. Rosenblatt takes the other view: nothing is stored as a picture. Instead, experience changes how strongly cells connect, and a familiar face simply flows down the strengthened paths to the right reaction. Recognising is the flowing; there is no separate look-up step.
He also has a practical point. Fifteen years earlier, McCulloch and Pitts described brains as exact logic circuits. Real brains are messy and partly random. So Rosenblatt builds his machine with random wiring on purpose, and studies it with probability instead of logic.
TakeawayIn a perceptron, memory lives in connection strengths, not in stored copies of what was seen.
Check yourself
- Why did Rosenblatt use random wiring?
- Because real brains are not wired exactly. He wanted to show learning could work even without a precise design, and he analysed it with probability.
Idea · pp. 389–392 · Figs. 1–3
Sensors, a random crowd, and competing answers
The machine looks at a small grid of light sensors, its "retina". Each sensor is on if light hits it. Behind the retina sits a crowd of association cells. Each one is wired to a handful of randomly chosen sensors, and it switches on if enough of its sensors are lit. Behind the crowd are two answer cells, say "Answer 1" and "Answer 2". Each answer cell listens to its own random group of association cells, and the louder group wins.
The random wiring never changes. What changes is how much each association cell's voice counts. Rosenblatt calls this its value. When you tell the machine "that was Answer 1", every association cell that was on and belongs to Answer 1's group gets a bit louder. Next time a similar picture lights up many of the same cells, Answer 1 wins.
Try it: draw a shape, train it, then test it on a shifted or slightly different shape.
TakeawayRandom fixed wiring turns a picture into a pattern of active cells. Learning only adjusts how much each cell counts toward each answer.
Check yourself
- Which part of the perceptron learns?
- Only the "values" of the association cells, that is, how strongly they push on the answers. The sensor-to-association wiring stays random and fixed.
- Why can it recognise a shifted bar it never saw?
- A shifted bar still lights up many of the same association cells as the trained bar, because each cell samples scattered sensors. Shared active cells carry the learned values over.
Idea · pp. 396–402
Real categories make generalising possible
Rosenblatt runs two thought experiments. In the first, the pictures are pure noise, random dots, and you label half "1" and half "2" for no reason. The machine can memorise some of them, but the more it learns, the more it forgets, and on new noise it guesses at chance. There is nothing to generalise from.
In the second, pictures come in real groups, like squares and circles. Now the members of a group light up overlapping cells, and learning one square helps with the next. The machine gets new examples right as often as old ones, and adding more association cells pushes it toward 100%.
TakeawayA learning machine can only generalise if the data has real structure: things in the same category must look more alike than things in different ones.
Check yourself
- What happens on random, meaningless labels?
- It memorises a bit, then gets worse as it learns more, and does no better than chance on new examples.
Idea · Later form (1962)
The learning rule as it is taught today
The 1958 paper describes learning in terms of cells gaining value. The tidy version every textbook now shows came from Rosenblatt's 1962 book: give each input a weight; add up weight × input; answer "yes" if the total is above zero. When the answer is wrong, nudge the weights toward the right answer. When it is right, leave them alone.
Geometrically, the weights draw a straight line through the data, and every mistake tilts the line. Press "Step" to watch it learn. Then switch to the "crossed" data, where no straight line can separate the two colours: the line keeps tilting forever. That limitation became the subject of a famous 1969 book.
TakeawayA perceptron draws one straight boundary and fixes it by nudging after each mistake. It succeeds when a straight boundary exists and fails forever when one doesn't.
Check yourself
- When does the perceptron change its weights?
- Only after a mistake. Correct answers leave the weights unchanged.
- Why can't it learn the crossed data?
- No single straight line separates the colours. Every fix for one point breaks another.
Idea · pp. 404–405
Where it stops: relations
Rosenblatt is honest about the limits. His perceptron can learn "say square when you see a square". It cannot learn "name the object to the left of the square", or "point to whatever came before the circle". Those answers depend on a relationship between things, not on which cells light up.
He concludes that recognising relations would need a system "more advanced in principle". That turned out to be right: it took networks with several learning layers, trained by backpropagation, to learn their own internal features.
TakeawayA single layer of learning handles concrete patterns. Relations in space and time need more.
The paper in four sentences
Memory is a change in connection effectiveness that biases responses, not a stored representation (the connectionist position of Hebb and Hayek).
A photo-perceptron maps a retina of S-points through randomly connected association units (A-units) to mutually inhibiting response units (R-units). Learning raises the "value" V of A-units active when a response is reinforced.
Because wiring is random, performance is predicted from two probabilities: Pa, the expected fraction of A-units a stimulus activates, and Pc, the conditional probability that an A-unit active for one stimulus is active for another.
In an "ideal" (random) environment the system memorises but cannot generalise. In a "differentiated" environment recall and generalisation converge to an asymptote above chance that approaches 1 as NA grows, provided Pc12 < Pa < Pc11. Psychological Review 65(6), 386–408.
Section · pp. 388–392 · Figs. 1–3
Organisation and dynamics
The general photo-perceptron has a retina of S-points; a projection area AI whose units take origin points from local clusters on the retina (connections falling off with distance); an association area AII whose units take origin points at random from AI; and responses R1…Rn, each receiving from a random source-set of A-units. Everything is feed-forward except the last stage, where responses feed back, in most of the analysis by inhibiting the A-units outside their own source-set (and hence rival responses).
An A-unit with x excitatory and y inhibitory origin points fires when the algebraic sum of its inputs reaches threshold: a = e + i ≥ θ. For the analysis Rosenblatt drops AI (the "simplified perceptron"), so A-units sample the retina directly and respond to overlapping lit area rather than contour.
A response proceeds in two phases (Fig. 3): a brief predominant phase in which scattered A-units in all source-sets fire, then a postdominant phase in which one response wins and suppresses the others' source-sets. Winner selection is by the sum (Σ-system) or mean (μ-system) of the values of active units in each source-set.
Section · pp. 391–392 · Table 1
Value dynamics: α, β and γ systems
The value V of an A-unit (amplitude, frequency, latency or transmission probability: the only new hypothetical construct) changes with reinforcement under one of three rules:
| System | Active A-units | Inactive A-units | Total value |
|---|---|---|---|
| α uncompensated gain | +1 per unit time active, retained | unchanged | grows with each reinforcement |
| β constant feed | each source-set receives K per unit time, shared in proportion to activity | share too | grows steadily |
| γ parasitic gain | gain value… | …drawn from inactive units of the same source-set | constant |
μ-systems generally beat Σ-systems because means are robust to random differences in source-set size; in the γ-system the two coincide. The γ-system performs best (Fig. 10); β worst, since uncontrolled growth amplifies chance differences.
Section · pp. 392–394 · Eqs. 1–2 · Figs. 4–6
Pa and Pc
Pa is computed by summing over the binomial (in the paper, Poisson-approximated) counts of excitatory and inhibitory origin points that fall inside a stimulus of area R, keeping terms with e − i ≥ θ. Pc conditions further on overlap with a second stimulus. Qualitatively: raising θ or y lowers Pa and lowers Pc more sharply; balanced excitation and inhibition makes Pa nearly independent of stimulus size; and Pc > 0 even for disjoint stimuli because one A-unit can collect origin points in both.
The figure estimates both quantities by Monte Carlo for a 20×20 retina, two square stimuli and adjustable overlap.
Section · pp. 394–402 · Eq. 4 · Figs. 7–11
Recall, generalisation and the two environments
After a learning series the system is frozen and tested. Pr is the probability of the reinforced response to a previously seen stimulus; Pg the probability of the correct class response to a new member of a learned class. Both take the form
with nsr stimuli learned per response, Ne the A-units not shared with the rival source-set, and c2 = 0 for Pg. In the ideal environment (random stimuli, arbitrary labels) c1 = 0: Z → 0 as nsr grows, so Pr decays toward 0.5 and Pg = 0.5 throughout. In a differentiated environment c1 > 0 and Pr, Pg converge to a common asymptote, above chance if Pc12 < Pa < Pc11, approaching 1 as NA increases.
Check yourself
- Why does Pr decay in the ideal environment?
- Each new random association adds value changes uncorrelated with earlier ones; with c1 = 0 the signal term stops growing while variance grows, so Z → 0 and Φ(Z) → 0.5.
- State the generalisation condition in words.
- Within-class pairs must share more active A-units than a random stimulus activates, and between-class pairs fewer.
Section · pp. 402–405
Extensions and limits
- Binary coding: replace n mutually exclusive responses with log2n response pairs (100 classes → 7 pairs).
- Bivalent reinforcement: values can fall as well as rise, enabling trial-and-error (reward/punishment) learning; simulated on an IBM 704 consistent with theory.
- Spontaneous concept formation: with value decay proportional to magnitude and reinforcement of whatever response occurs, the system partitions two dissimilar classes without supervision.
- Distributed memory: removing A-units degrades all associations mildly rather than deleting particular ones.
- Limit: relational judgements ("the object left of the square") are beyond statistical separability.
Rosenblatt's methodological claim: performance is predicted from six measurable physical parameters (x, y, θ, ω, NA, NR), giving parsimony, falsifiability and generality compared with Hull-style learning theories.
Section · Beyond the paper
Modern reading and later refinements
The simplified perceptron is a layer of fixed random threshold features followed by a trained linear readout, close in spirit to later random-feature methods and extreme learning machines. Only the readout learns.
The error-correction rule and the convergence theorem are from Principles of Neurodynamics (1962); Block (1962) and Novikoff (1962) gave convergence proofs. Novikoff's bound: if the data are separable with margin γ and ‖x‖ ≤ R, the number of mistakes is at most (R/γ)2.
Reference
Glossary
- S-point
- Retinal sensory unit, all-or-none.
- A-unit
- Association unit; fires if excitatory minus inhibitory input ≥ θ.
- R-unit
- Response unit; receives from its source-set and inhibits rivals.
- Source-set
- The A-units connected to a given response.
- Value V
- Effectiveness of an A-unit's output; the only learned quantity.
- α, β, γ
- Value-change rules: unlimited gain, constant feed per source-set, gain at inactive units' expense.
- Σ / μ system
- Winner chosen by sum vs mean of active values.
- Pa, Pc
- Fraction of A-units activated; conditional co-activation for two stimuli.
- Pr, Pg
- Probability of correct recall; of correct generalisation.
- Bivalent
- Reinforcement that can raise or lower values.