The paper in four sentences
You can teach a large language model a task just by putting a few solved examples in front of the question ("few-shot prompting"), but for problems that need several steps, showing only the final answers works poorly.
If the examples instead include the reasoning in plain words ("Roger started with 5 balls. 2 cans of 3 tennis balls each is 6…"), the model writes similar reasoning before its own answer, a "chain of thought".
This helped only very large models (around 100 billion parameters and up); smaller models wrote fluent but wrong reasoning and sometimes got worse.
With eight such examples, the 540-billion-parameter PaLM model solved 56.9% of grade-school maths problems (up from 17.9%), beating the previous best, a fine-tuned GPT-3 with a separate answer checker. A few months later, others showed that simply adding "Let's think step by step" also works.
Idea · §1
Teaching by example, inside the prompt
A language model continues whatever text it is given. GPT-3 showed you can exploit this to "teach" a task without any retraining: write a few example questions with their answers, then the new question, and the model continues with an answer in the same style. This is few-shot prompting; the examples are called exemplars.
It works well for simple tasks but badly for problems needing several steps of reasoning, like maths word problems. And making models bigger helped less on these than on other tasks; the curve was nearly flat. Older approaches trained models specifically to write out solutions, which needs large datasets of worked solutions for every task.
TakeawayFew-shot prompting teaches a task through examples in the prompt, but answer-only examples don't teach multi-step reasoning.
Idea · §2 · Fig. 1
Show the working
The paper's idea is simply to put the reasoning into the exemplars. Each example now goes question, then a short chain of reasoning in plain language, then the answer. Faced with a new question, the model imitates the pattern: it writes its own chain of reasoning, then an answer.
Toggle between the two prompts below. These are the paper's own example and the model's actual outputs from its Figure 1.
TakeawayWrite the reasoning into the examples and a large model will write reasoning into its answers.
Check yourself
- Was the model retrained or changed in any way?
- No. Only the prompt changed. Chain-of-thought prompting works with an off-the-shelf model, which is one of its main attractions.
- Why might writing intermediate steps help a model that produces one word at a time?
- Each step it writes becomes part of the input for the next words, so it can use earlier results (like "23 − 20 = 3") instead of having to jump to the final number in one go. It also spends more computation on harder problems, since longer problems get longer chains.
Idea · §3.2 · Fig. 4, Table 2
Only big models can do it
The authors tried three families of models at many sizes. For small models, chain-of-thought prompting did nothing or made things worse: they produced reasoning that read smoothly but didn't make sense. Only at around 100 billion parameters did the benefit appear, and then it was large.
The authors describe this as an emergent ability of scale: something that is absent in small models and appears in large ones, rather than improving gradually. Pick a family below to see the paper's numbers on GSM8K, a set of grade-school maths word problems.
TakeawayChain of thought is useless or harmful for small models and dramatically helpful for very large ones.
Check yourself
- What happens to the smallest GPT-3 model with chain-of-thought prompting?
- It gets worse: 2.2% with standard prompting, 0.5% with chain of thought. Small models imitate the form of reasoning without being able to do it.
Idea · §4–5 · Tables 4–5
Not just arithmetic
The same trick helped on commonsense questions. For example, on deciding whether a sports sentence is plausible ("Joao Moutinho caught the screen pass in the NFC championship"; he is a footballer, so no), PaLM 540B went from 80.5% to 95.4%, above the 84% of an unaided sports enthusiast. On StrategyQA, questions like "Would a pear sink in water?" that need several implicit steps, it went from 68.6% to 77.8%.
It also helped on symbolic puzzles such as "take the last letter of each word and join them". Most strikingly, after seeing examples with only two words, models using chain of thought could handle three or four words, which standard prompting failed at almost completely. Writing the steps lets the model extend a procedure to longer inputs.
TakeawayChain of thought helps on any task a person would solve by reasoning in words, and helps models generalise to longer problems than in the examples.
Idea · §6 and later
Is it really reasoning? And what came next
The authors are careful: the chains look like human reasoning, but whether the model is truly "reasoning" is left open. Chains can also be wrong while the final answer is right (especially on multiple-choice questions), and the method only helps models that are expensive to run.
TakeawayPrompting for step-by-step reasoning started a line of work that led to models trained to think before they answer.
Abstract, restated
Generating a chain of thought, a series of intermediate reasoning steps, significantly improves the ability of large language models to perform complex reasoning. Such abilities emerge naturally in sufficiently large models through chain-of-thought prompting, where a few chain-of-thought demonstrations are provided as exemplars. Experiments on three large language models show improvements on arithmetic, commonsense and symbolic reasoning tasks. Prompting PaLM 540B with eight chain-of-thought exemplars achieves state-of-the-art accuracy on GSM8K, surpassing even fine-tuned GPT-3 with a verifier. Wei, Wang, Schuurmans, Bosma, Ichter, Xia, Chi, Le, Zhou (Google Research, Brain Team), NeurIPS 2022 (arXiv 2201.11903).
Section · §2
Definition and motivation
"A chain of thought is a series of intermediate natural language reasoning steps that lead to the final output." A chain-of-thought prompt is a few-shot prompt of ⟨input, chain of thought, output⟩ triples. The authors list four properties: (1) decomposition, so additional computation can be allocated to problems requiring more steps; (2) an interpretable window into model behaviour, suggesting how an answer was reached and where it went wrong (with the caveat that full characterisation of the underlying computation remains open); (3) applicability, in principle, to any task humans can solve via language; (4) it can be elicited from sufficiently large off-the-shelf models simply by including exemplars, with no fine-tuning.
The approach unites two older ideas: rationale-augmented training and fine-tuning for intermediate steps (e.g. Ling et al., 2017; Cobbe et al., 2021), which requires costly annotated datasets, and in-context few-shot prompting (Brown et al., 2020), which works poorly on reasoning and scales flatly. The paper's motivation is to keep the second's convenience while getting the first's benefit.
Section · §3.1–3.2 · Tables 1–2, Fig. 4
Arithmetic reasoning
Setup. Benchmarks: GSM8K, SVAMP, ASDiv, AQuA, MAWPS. Eight hand-written chain-of-thought exemplars used for all benchmarks except AQuA (four exemplars from its training set, since it is multiple choice). Models: GPT-3 (350M, 1.3B, 6.7B, 175B; the 175B results use text-davinci-002), LaMDA (420M–137B), PaLM (8B, 62B, 540B), plus UL2 20B and Codex (code-davinci-002). Greedy decoding. LaMDA results average five random seeds of exemplar order.
| GSM8K (Table 2) | Standard | Chain of thought | Δ |
|---|---|---|---|
| PaLM 8B | 4.9 | 4.1 | −0.8 |
| PaLM 62B | 9.6 | 29.9 | +20.3 |
| PaLM 540B | 17.9 | 56.9 | +39.0 |
| LaMDA 137B | 6.5 | 14.3 | +7.8 |
| GPT-3 175B | 15.6 | 46.9 | +31.3 |
| Prior best (fine-tuned GPT-3 + verifier) | 55 | ||
Three findings (§3.2): CoT is an emergent ability of scale, with no positive effect for small models (Appendix A.1: it hurts most models under 10B); gains are larger for harder problems (GSM8K, where baseline is lowest, more than doubles for the largest GPT and PaLM) and small or negative on easy one-step sets (e.g. MAWPS SingleOp); and PaLM 540B with CoT is competitive with or beats prior fine-tuned state of the art (SVAMP 79.0 vs prior best 57.4; MAWPS 93.3 vs 88.4).
Error analysis. Of 50 correct LaMDA 137B GSM8K answers, almost all had logically and mathematically valid chains (the main text and Appendix D.1 differ slightly on the count, at most two being correct by chance). Of 50 incorrect ones, 46% had minor errors (8% calculator-only errors, 16% symbol-mapping errors, 22% one step missing) and 54% major errors in semantic understanding or coherence. Scaling PaLM from 62B to 540B fixed a large share of one-step-missing (12 of 18) and semantic-understanding errors (6 of 20).
Section · §3.3–3.4 · Figs. 5–6, Table 6
Ablations and robustness
Is it the natural-language reasoning that helps, or something incidental? Three ablations isolate alternatives, and several robustness checks vary who wrote the chains. Numbers below are LaMDA 137B on GSM8K (mean ± std over seeds, Table 6); the paper's Figure 5 shows the same pattern for PaLM 540B.
Conclusions: an equation alone is not enough on semantically hard problems (it helps on easier one- or two-step sets, per the paper); extra tokens alone (dots matching the chain's length) do not help, so the gain is not just more computation; reasoning placed after the answer does not help, so the benefit is not merely that CoT activates relevant knowledge. CoT is robust to annotator style and exemplar choice, though variance exists (on coin flip, accuracy ranged from 99.6% with annotator A to 71.4% with annotator C, Appendix A.2).
Section · §4–5 · Tables 4–5, Figs. 7–8
Commonsense and symbolic reasoning
| PaLM 540B (Table 4) | Standard | CoT | Note |
|---|---|---|---|
| CSQA | 78.1 | 79.9 | small gain |
| StrategyQA | 68.6 | 77.8 | prior best 69.4 |
| Date understanding | 49.0 | 65.3 | BIG-bench |
| Sports understanding | 80.5 | 95.4 | sports enthusiast 84 |
| SayCan (robot planning) | 80.8 | 91.7 | maps to robot actions |
(The text gives StrategyQA as 75.6% while Table 4 lists 77.8; the table is used here.) Symbolic tasks test out-of-domain length generalisation: exemplars use 2 words or 2 coin flips, test sets also use 3 and 4. Build a last-letter-concatenation chain below and compare with the paper's results.
Coin flip (does a coin, initially heads up, remain heads up after people flip or don't flip it?): 2 flips 98.1 → 100.0 (in domain), 3 flips 49.3 → 98.6, 4 flips 54.8 → 90.2 (out of domain).
Section · §6 and later work
Limitations and lineage
- Reasoning or imitation? Emulating a thought process does not establish that the network is "reasoning"; this is left open.
- Unfaithful chains. No guarantee of correct reasoning paths; correct answers can come from wrong chains, especially for multiple-choice or binary tasks. Later work (e.g. Turpin et al., 2023) showed stated chains can omit factors that actually drive the answer.
- Cost. Emergence only at ~100B parameters makes serving expensive; inducing reasoning in small models is future work.
- Annotation. Cheap for few-shot prompting, but costly if used for fine-tuning; exemplar sensitivity exists.
Check yourself
- Which ablation rules out "CoT just gives the model more tokens to compute with"?
- Variable compute only: replacing the chain with a sequence of dots of equal length gave 6.4% vs 6.5% standard, no gain.
- Which ablation rules out "CoT just activates relevant knowledge"?
- Chain of thought after the answer: the reasoning is present in the exemplars but produced after the answer, and it gives 6.1%, no better than standard.
Reference
Glossary
- Few-shot prompting
- Including input–output exemplars in the prompt, with no weight updates.
- Exemplar
- One worked example in the prompt.
- Chain of thought
- Intermediate natural-language reasoning steps before the answer.
- Emergent ability
- One absent in smaller models and present in larger ones.
- Greedy decoding
- Always taking the most likely next token.
- Out-of-domain (OOD)
- Test inputs longer or otherwise different from exemplars.
- Self-consistency
- Majority vote over answers from sampled chains.
- GSM8K
- 8.5k grade-school maths word problems (Cobbe et al., 2021).