The paper in four sentences
A common way to teach robots is by demonstration: a person controls the robot a few hundred times and the robot learns to copy what it sees ("behaviour cloning").
Demonstrations often solve a task in more than one way, and standard learners blur those ways together, producing actions that match none of them.
Diffusion Policy treats the next stretch of robot motion like an image to be generated: start from random noise and refine it step by step, guided by what the cameras see. Like an image generator, it can produce any of the valid options, cleanly.
It plans about a second of motion at a time, carries out part of it, then plans again. This made it markedly more reliable in simulation and on real robots, and the same idea now sits inside large robot models such as π0.
Idea · §1, §4.1 · Fig. 3
The averaging trap
In behaviour cloning, a robot is shown many examples of "what the cameras saw" paired with "what the person made the robot do", and a neural network learns to predict the action from the observation. The simplest version predicts a single best action, trained to be as close as possible to the demonstrated ones.
That works when there is one right answer. But suppose half the demonstrations go left around a cup and half go right. The prediction that is "closest on average" to both is straight down the middle, into the cup. In the jargon, the action distribution is multimodal (it has several separate peaks), and a single averaged prediction lands between them.
Below, the dashed lines are demonstrations of getting from the bottom to the top around an obstacle. Compare the averaged prediction with samples from a diffusion model trained on the same demonstrations.
TakeawayWhen demonstrations solve a task in several ways, predicting the average action can be worse than any of them. The learner must be able to choose one way.
Check yourself
- Why does a squared-error learner end up in the middle?
- For a given observation, the single prediction that minimises the average squared distance to all demonstrated actions is their mean, which here lies between the two routes.
- What does "multimodal" mean here?
- For the same situation, there are several distinct good actions (modes), such as going left or going right.
Idea · §2 · Fig. 1
Robot motion, generated like an image
Image diffusion models generate a picture by starting from random static and removing noise step by step. Diffusion Policy applies the same procedure to robot actions. The thing being generated is not a picture but a short action sequence: the next 16 target positions for the robot's hand, say. The noise-removing network is also given what the robot currently sees, so the actions it generates fit the situation.
Because a diffusion model learns the whole spread of possible outputs, not just their average, each run lands on one of the demonstrated ways. Which one depends on the random starting noise. And because the whole sequence is generated together, it is a coherent plan rather than a jumble of steps from different strategies.
Drag the slider to watch one action sequence emerge from noise.
TakeawayDiffusion Policy generates a whole sequence of future actions by denoising, conditioned on what the robot sees, so it can represent several valid strategies and commit to one.
Check yourself
- In Diffusion Policy, what plays the role of the "image"?
- A short sequence of future robot actions (for example, target positions for the next 16 time steps).
- What decides whether a given run goes left or right?
- The random noise it starts from (and the randomness added during denoising). Different noise leads to different valid plans.
Idea · §2.3, §4.3 · Figs. 2, 5
Plan a stretch, do part of it, plan again
Generating takes time, many network passes per plan, so the robot can't afford to plan from scratch at every tiny time step. And committing to a very long plan would make it slow to react if something changes. The paper's compromise, borrowed from control engineering, is receding-horizon control: look at the last couple of camera frames, plan the next 16 steps, carry out the first 8, then look again and make a new plan.
Carrying out several steps of one plan has a second benefit: the robot sticks with one strategy instead of flip-flopping between "left" and "right" at every moment. The paper found 8 executed steps to be the best setting for most of its tasks; fewer gave jittery motion, more made the robot slow to react.
TakeawayDiffusion Policy plans 16 steps, executes about 8, and re-plans: a balance between consistency and reacting to change.
Idea · §5–7 · Tables 1–6
Results in simulation and on real robots
The authors compared Diffusion Policy with three strong earlier methods: a recurrent network that outputs a mixture of bell curves (LSTM-GMM), an "implicit" energy-based policy (IBC), and a transformer that clusters actions (BET). Across 12 simulated tasks (15 in v5) from four benchmarks, from lifting objects to multi-stage kitchen chores, it improved on the best of them by 46.9% on average.
On real robots: pushing a T-shaped block into place (95% success, the baselines 0 to 20%), flipping a mug (90% vs 0% for LSTM-GMM), and pouring and spreading sauce on pizza dough. An extended version adds two-armed tasks such as unrolling a mat and folding a shirt (75% success on each).
Two practical findings stood out. Commanding target positions worked better than commanding velocities, the opposite of what earlier methods preferred. And training was stable: unlike the energy-based method, its performance did not swing wildly between training checkpoints.
TakeawayDiffusion Policy outperformed the best previous behaviour-cloning methods by a wide margin, in simulation and on real hardware.
Idea · §9 and legacy
Limits, and where the idea went
Diffusion Policy still only imitates: it needs good demonstrations and can't improve beyond them on its own. And generating each plan takes many network passes, which makes it slower and more computationally demanding than simpler policies, an issue for very fast control loops.
TakeawayTreating robot actions as something to generate, not just predict, handled multiple strategies cleanly and shaped how today's robot foundation models produce actions.
Abstract, restated
Diffusion Policy generates robot behaviour by representing a visuomotor policy as a conditional denoising diffusion process. Benchmarked on 12 tasks from 4 robot manipulation benchmarks (15 in v5), it consistently outperforms existing state-of-the-art robot learning methods, with an average improvement of 46.9%. It learns the gradient of the action-distribution score function and iteratively optimises with respect to that gradient field at inference via a series of stochastic Langevin dynamics steps. The diffusion formulation handles multimodal action distributions, suits high-dimensional action spaces, and trains stably. Key technical contributions are receding-horizon control, visual conditioning, and a time-series diffusion transformer. Chi, Xu, Feng, Cousineau, Du, Burchfiel, Tedrake and Song; RSS 2023, with an extended version (arXiv:2303.04137v5, 2024) adding bimanual experiments.
Section · §2 · Eqs. 1–5
Formulation
Starting from xK ~ N(0, I), K denoising iterations produce x0:
which the paper reads as noisy gradient descent x′ = x − γ∇E(x) (Eq. 2): εθ predicts the gradient field of an implicit energy, and α, γ, σ as functions of k are the noise schedule. Training is DDPM's (Eq. 3). For a policy, the generated variable is the action sequence At and the denoiser is conditioned on observations Ot:
Modelling p(At | Ot) rather than the joint p(At, Ot) (as Diffuser, Janner et al. 2022, does for planning) avoids predicting future observations, speeds inference and makes end-to-end training of the vision encoder feasible. Horizons (§2.3, Fig. 2a): the policy takes the last To observations, predicts Tp actions and executes Ta of them before re-planning, warm-starting the next inference from the previous prediction. Typical values (Table 7): To = 2, Tp = 16, Ta = 8. Actions are normalised to [−1, 1] (App. A).
Section · §3 · Fig. 2b–c · Table 5
Key design decisions
- CNN-based denoiser. The 1-D temporal convolutional U-Net from Diffuser, with observation features and the step k injected by FiLM (feature-wise scale and shift) in every convolutional layer, and only the action sequence as the denoised signal. Works out of the box on most tasks but is biased towards low-frequency signals, so it struggles with rapidly changing actions such as velocity commands. The authors recommend starting here.
- Time-series diffusion transformer. A minGPT-style decoder: noisy action tokens as inputs, the embedding of k and the observation features attended to by causal cross-attention. Better on high-rate and velocity-control tasks but more sensitive to hyperparameters.
- Visual encoder. A ResNet-18 trained from scratch end to end (one per camera view), with global average pooling replaced by spatial softmax pooling and BatchNorm replaced by GroupNorm for stability with EMA weights. In Table 5 (Square task), from-scratch ResNet-18 scored 0.94 versus 0.58 frozen-pretrained; a fine-tuned CLIP ViT-B/16 reached 0.98.
- Noise schedule. The square cosine schedule of Improved DDPM worked best (§3.3).
- Inference speed. DDIM decouples training and inference iterations: 100 training iterations and 10 inference iterations gave about 0.1 s latency on an Nvidia 3080 (§3.4); real-world experiments used 16 inference iterations (App. A.4).
Section · §4 · Eqs. 6–8 · Figs. 3–6
Why it works: four properties
Multimodality (§4.1, Fig. 3). Stochastic initialisation and Langevin-style sampling let different rollouts converge to different modes, while sequence prediction keeps each rollout in one mode. On Push-T, Diffusion Policy commits to going left or right around the block; LSTM-GMM and IBC are biased to one mode, and BET fails to commit, lacking temporal consistency.
Position control (§4.2, Fig. 4). Diffusion Policy works best with absolute position actions, whereas the baselines did best with velocity control. Explanation: position actions are more multimodal (a problem for the baselines, not for diffusion) and suffer less from compounding error.
Action sequences (§4.3, Fig. 5). Per-step multimodal sampling can alternate between modes from one step to the next, giving jittery actions; predicting and executing a sequence prevents that. Sequence prediction also helps with idle pauses in demonstrations, which single-step policies tend to overfit (BC-RNN and IBC "often get stuck" in real experiments). Too long a horizon reduces performance through slow reaction; Ta = 8 was best on most tasks, and position control stayed near peak performance with up to 4 steps of latency.
Training stability (§4.4, Eqs. 6–8). IBC models pθ(a | o) = e−Eθ(o,a)/Z(o, θ) and trains with an InfoNCE-style loss whose negative samples estimate the intractable Z, causing unstable training (Fig. 6: its success rate oscillates even as the loss falls smoothly). Diffusion avoids Z entirely, because the score does not depend on it:
The figure is a toy version of the §4.3 argument. A point robot heads to a goal past an obstacle; at each re-plan the policy samples a "left" or "right" detour, with the choice depending on its current lateral position (a multimodal policy conditioned on observation). It then executes Ta steps of that plan.
Section · §5–7 · Tables 1–6
Evaluation
Simulation benchmarks: Robomimic (Lift, Can, Square, Transport, ToolHang; proficient-human "ph" and mixed-human "mh" data), Push-T, Multimodal Block Pushing, and Franka Kitchen, with state and image observations. Baselines: LSTM-GMM (BC-RNN), IBC, BET. Metrics are the max and the average of the last 10 checkpoints (each over many rollouts); DP-C and DP-T denote the CNN and transformer variants. The 46.9% is the average over tasks of the relative improvement of the best Diffusion Policy variant over the best baseline, computed as described in App. B.2.
| Task (max / avg) | LSTM-GMM | IBC | BET | DP-C | DP-T |
|---|---|---|---|---|---|
| Push-T, state (Table 1) | 0.67 / 0.61 | 0.90 / 0.84 | 0.79 / 0.70 | 0.95 / 0.91 | 0.95 / 0.79 |
| Push-T, image (Table 2) | 0.69 / 0.54 | – | – | 0.91 / 0.84 | – |
| Kitchen p4 (Table 4) | 0.34 | – | 0.44 | 0.99 | 0.96 |
| Block push p2 (Table 4) | – | – | 0.71 | – | 0.94 |
(Cells marked – are omitted here, not missing from the paper.) Real world (§6–7). Push-T (Table 6): 0.95 success, 0.80 IoU, against 1.00 / 0.84 for a human and 0 to 0.20 success for IBC and LSTM-GMM; robust to occlusion and to the block being moved mid-task (Fig. 8). Mug flipping (Fig. 9): 90% over 20 trials, LSTM-GMM 0%. Sauce pouring IoU 0.74 and spreading coverage 0.77 (Fig. 10). Bimanual (§7): egg beater 55% (210 demos), mat unrolling 75% (162), shirt folding 75% (284). Diffusion Policy also beats LSTM-GMM at every training-set size tested (Fig. 15).
Section · §9 and lineage
Limitations and lineage
- Behaviour cloning limits. Needs adequate demonstrations; suboptimal or negative data are not used. Combining with reinforcement learning is left to future work.
- Compute and latency. Higher than LSTM-GMM; action chunking only partly compensates for high-rate control. The authors point to faster samplers and consistency models.
- Per-task training. Each policy is trained for one task from its own demonstrations, without language or broad pretraining.
Reference
Glossary
- Behaviour cloning
- Supervised learning of a policy from demonstration (observation, action) pairs.
- Multimodal distribution
- A distribution with several separated peaks (several valid actions).
- To, Tp, Ta
- Observation, prediction and action-execution horizons.
- Receding horizon
- Plan over a horizon, execute the first part, re-plan from the new state.
- FiLM
- Feature-wise linear modulation: conditioning by a learned per-channel scale and bias.
- IBC
- Implicit behavioural cloning: an energy-based policy trained with negative samples.
- LSTM-GMM / BET
- Recurrent policy with a Gaussian-mixture output; Behaviour Transformer with k-means action bins plus offsets.
- DDIM
- Deterministic diffusion sampler allowing fewer inference than training steps.