Study guide · Neural networks · 1969

Perceptrons: An Introduction to Computational Geometry

Marvin Minsky, Seymour Papert · MIT Press (book)

Two MIT researchers turn the perceptron from an exciting demo into a subject of mathematics, and prove precise limits on what it can recognise. Some things that look easy to us, like "is this shape all one piece?", turn out to be impossible for a perceptron whose parts each look at only a small region.

The book in four sentences

A perceptron makes its decision by adding up votes from many small detectors, each looking at part of the picture.

Minsky and Papert ask: which questions about a picture can be answered that way, if every detector sees only a small patch?

Some can (is the shape convex?), but some provably can't: whether the number of dots is even or odd, and whether a shape is connected in one piece. For those, some detector must see the whole picture, or grow with it.

The book was read as a verdict against neural networks and is often blamed for a lull in the field, though historians disagree about how much it caused.

About the sourceThe book has no free legal copy online, so the archive links to a free explainer. This guide describes the book's main results as they are standardly reported; chapter references are to the 1969 edition.

Idea · Chapter 0

A perceptron is a committee of small detectors

Picture a large committee judging a photo. Each member only gets to look at one small piece through a peephole, and can only report a simple yes or no about that piece. Each member's vote has a weight. The chair adds up the weighted votes and announces "yes" if the total is big enough.

That is a perceptron, as Minsky and Papert define it. The interesting question is: which yes/no questions about the whole photo can such a committee answer, when every peephole is small?

Everyday pictureAsking a hundred people, each looking at one tile of a mosaic, whether the mosaic shows a single unbroken line. None of them can see far enough to know, and simply adding up their opinions doesn't help.

TakeawayThe book studies a perceptron as weighted voting by limited-view detectors, and asks what that structure can and cannot decide.

Idea · Chapter 3

Even or odd: the parity problem

Here is a question a child can answer: is the number of lit squares even or odd? Yet no committee of small peepholes can do it. The reason is simple: if even one square is hidden from you, the answer is a coin flip, because that one square can flip it either way. Parity depends on every square at once.

Minsky and Papert proved that a perceptron for parity needs at least one detector that sees the entire picture. Try it below: hide some squares and see that the visible ones tell you nothing.

Click squares to switch them on. Hidden squares (dashed) are outside the detector's view. Unless it sees all 8, the possible pictures it can't tell apart are exactly half even and half odd.

TakeawayParity can only be decided by looking at everything at once, so a perceptron built from small detectors cannot compute it.

Check yourself
If you can see 7 of 8 squares, what do you know about parity?
Nothing. The hidden square makes it even or odd with equal likelihood.

Idea · Chapters 5 and 8

"Is it one piece?" can't be decided locally

The book's most famous argument concerns connectedness: is a shape all one piece? Imagine four long strands with their ends tied together in pairs. At each end there are two ways to tie them. Depending on the combination, you get one big loop or two separate loops.

The trick: a detector with a small view sees only one end, or only the middle. Whichever way the left end is tied, the right end decides the outcome, and vice versa. Set the committee's votes below and try to make it get all four cases right. You can't, and the box shows the short proof why.

Left end: Right end:

Middle detectors see the same thing in all four cases, so they only add a constant (folded into the threshold). Why no setting works: the two "one loop" cases need L₀+R₁ > t and L₁+R₀ > t; adding gives L₀+L₁+R₀+R₁ > 2t. The two "two loops" cases need L₀+R₀ < t and L₁+R₁ < t; adding gives the same sum < 2t. Both can't hold.

TakeawayWhen the answer depends on how distant parts relate, adding up local evidence fails. Connectedness is such a question.

Check yourself
Why can't the left-end detectors help?
"Side by side" on the left gives one loop in one case and two loops in the other, depending on the right end. Seen alone, it is no evidence either way.

Idea · Chapter 13 and history

What the book did, and a popular myth

A common story says this book "proved neural networks can't do XOR" and killed the field. Both halves are too simple. XOR (one or the other but not both) is just parity on two inputs. A perceptron with one detector seeing both inputs computes it easily. And everyone, including the authors, knew that networks with more layers could compute anything. What the book proved is narrower: perceptrons whose detectors see only a limited part of the input can't compute certain global properties.

What the authors added was a guess: that adding layers would not lead anywhere useful, because nobody knew how to train them. That guess turned out wrong once backpropagation made training hidden layers practical. Funding for neural-network research did shrink in the 1970s, but historians still argue about how much the book caused it.

TakeawayThe theorems were correct and specific; the pessimism about multi-layer networks was a conjecture, and it did not hold up.

Follow the idea

Built on: Perceptron (1958)

Led to: Backpropagation (1986)