The book in four sentences
A perceptron makes its decision by adding up votes from many small detectors, each looking at part of the picture.
Minsky and Papert ask: which questions about a picture can be answered that way, if every detector sees only a small patch?
Some can (is the shape convex?), but some provably can't: whether the number of dots is even or odd, and whether a shape is connected in one piece. For those, some detector must see the whole picture, or grow with it.
The book was read as a verdict against neural networks and is often blamed for a lull in the field, though historians disagree about how much it caused.
Idea · Chapter 0
A perceptron is a committee of small detectors
Picture a large committee judging a photo. Each member only gets to look at one small piece through a peephole, and can only report a simple yes or no about that piece. Each member's vote has a weight. The chair adds up the weighted votes and announces "yes" if the total is big enough.
That is a perceptron, as Minsky and Papert define it. The interesting question is: which yes/no questions about the whole photo can such a committee answer, when every peephole is small?
TakeawayThe book studies a perceptron as weighted voting by limited-view detectors, and asks what that structure can and cannot decide.
Idea · Chapter 3
Even or odd: the parity problem
Here is a question a child can answer: is the number of lit squares even or odd? Yet no committee of small peepholes can do it. The reason is simple: if even one square is hidden from you, the answer is a coin flip, because that one square can flip it either way. Parity depends on every square at once.
Minsky and Papert proved that a perceptron for parity needs at least one detector that sees the entire picture. Try it below: hide some squares and see that the visible ones tell you nothing.
TakeawayParity can only be decided by looking at everything at once, so a perceptron built from small detectors cannot compute it.
Check yourself
- If you can see 7 of 8 squares, what do you know about parity?
- Nothing. The hidden square makes it even or odd with equal likelihood.
Idea · Chapters 5 and 8
"Is it one piece?" can't be decided locally
The book's most famous argument concerns connectedness: is a shape all one piece? Imagine four long strands with their ends tied together in pairs. At each end there are two ways to tie them. Depending on the combination, you get one big loop or two separate loops.
The trick: a detector with a small view sees only one end, or only the middle. Whichever way the left end is tied, the right end decides the outcome, and vice versa. Set the committee's votes below and try to make it get all four cases right. You can't, and the box shows the short proof why.
TakeawayWhen the answer depends on how distant parts relate, adding up local evidence fails. Connectedness is such a question.
Check yourself
- Why can't the left-end detectors help?
- "Side by side" on the left gives one loop in one case and two loops in the other, depending on the right end. Seen alone, it is no evidence either way.
Idea · Chapter 13 and history
What the book did, and a popular myth
A common story says this book "proved neural networks can't do XOR" and killed the field. Both halves are too simple. XOR (one or the other but not both) is just parity on two inputs. A perceptron with one detector seeing both inputs computes it easily. And everyone, including the authors, knew that networks with more layers could compute anything. What the book proved is narrower: perceptrons whose detectors see only a limited part of the input can't compute certain global properties.
What the authors added was a guess: that adding layers would not lead anywhere useful, because nobody knew how to train them. That guess turned out wrong once backpropagation made training hidden layers practical. Funding for neural-network research did shrink in the 1970s, but historians still argue about how much the book caused it.
TakeawayThe theorems were correct and specific; the pessimism about multi-layer networks was a conjecture, and it did not hold up.
Setting
Let R be a finite retina and Φ a family of predicates φ: 2R → {0,1}. A predicate ψ is linear with respect to Φ (a perceptron over Φ) if ψ(X) = ⟦Σφ∈Φ αφ φ(X) > θ⟧ for real α, θ. The order of ψ is the smallest k such that ψ is linear over some Φ whose members each depend on at most k points of R.
The book (Minsky & Papert, MIT Press 1969; expanded edition 1988 with a new prologue and epilogue but no new theorems) develops an algebraic theory (Part I), a geometric theory (Part II) and an analysis of learning (Part III).
Section · Ch. 1–2
Order, masks, and group invariance
Any predicate can be written as a linear threshold over masks φA(X) = ⟦A ⊆ X⟧, so order is equivalently the minimum degree of a polynomial threshold representation over {0,1}|R|. Order 1 predicates are ordinary linear threshold functions of the pixels.
The group invariance theorem is the main tool: if ψ is invariant under a group G of permutations of R, and Φ is closed under G, then if ψ is linear over Φ it is linear over Φ with coefficients constant on each G-orbit (average the coefficients over the group). This collapses a representation to a few free numbers, which then lead to contradictions.
Section · Ch. 3–4
Parity has order |R|
Parity is invariant under all permutations of R, so by group invariance a representation of order k can use equal coefficients for all masks of the same size j ≤ k. On an input with |X| = n, the weighted sum is then a polynomial P(n) = Σj≤k αj C(n, j) of degree ≤ k. For it to cross θ with every change of parity for n = 0…|R|, P − θ must change sign |R| times, so it needs degree ≥ |R|. Hence ord(ψPARITY) = |R|.
The same method shows "one-in-a-box" (every box of a partition contains at least one point) has order growing with box size, and gives the And/Or theorem: a conjunction or disjunction of two low-order predicates on disjoint parts of the retina can have high order.
Check yourself
- Why does symmetry reduce the representation to a polynomial in |X|?
- With coefficients equal within each mask size, the sum depends only on how many masks of each size are contained in X, which is C(|X|, j).
- Does order |R| mean parity is not a perceptron predicate?
- No. It is a perceptron predicate if a mask can see the whole retina. The point is that no bounded-order family suffices as R grows.
Section · Ch. 5, 8
Connectedness is not of finite order
For diameter-limited perceptrons (each φ depends only on points within a fixed diameter), the switching argument on the Plain English tab is a complete proof. Build four figures from a long central section and two end-pieces, each end chosen from two configurations, so that the figure is connected iff the ends differ. Each φ sees at most one end, so the weighted sum decomposes as C + L(a) + R(b). Connectedness requires
Summing the first pair and the second pair gives contradictory inequalities on the same total. For order-limited perceptrons (each φ depends on few points, but they may be far apart), Chapter 5 uses a reduction from one-in-a-box to show that the order of ψCONNECTED grows without bound with the size of the retina.
By contrast several geometric predicates have small order: convexity has order 3 (X is convex iff no pair of its points has its midpoint outside X, a condition on triples), and some predicates are order 1 or 2. Chapter 7 (stratification) builds bounded-order representations for some translation-invariant predicates, but only with enormously large coefficients.
Section · Ch. 10–13
Learning, coefficient size, and the multilayer conjecture
Part III restates the perceptron convergence theorem and compares perceptron learning with other procedures. It also shows that for some predicates the coefficients must grow exponentially or worse in |R|, so even when a representation exists it may be impractical to store, and learning it may take an impractically long time.
In the concluding chapter (§13.2) the authors consider multilayer extensions such as Gamba perceptrons, and state their "intuitive judgment" that extending the theory to multilayer systems would be "sterile". This was a conjecture about learnability, not a theorem.
Check yourself
- Which of the book's claims are theorems and which a conjecture?
- Order bounds for parity, one-in-a-box and connectedness, and the diameter-limited impossibility, are theorems. The sterility of multilayer extensions is a stated judgment.
Reference
Glossary
- Retina R
- The finite set of input points.
- Partial predicate φ
- A detector computing a yes/no function of part of the retina.
- Support
- The set of points a predicate depends on.
- Order
- Minimum over representations of the largest support size used.
- Mask
- φA(X) = 1 iff all points of A are in X; a conjunction.
- Diameter-limited
- Each φ's support fits in a disc of fixed diameter.
- Group invariance
- Averaging coefficients over a symmetry group preserves a representation.
- Gamba perceptron
- A two-layer variant whose first-layer units are themselves linear threshold units.