Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Authors: Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, Shuran Song

Published: 2023 (Journal Paper)

Source: International Journal of Robotics Research (IJRR)

Algorithm: Diffusion Policy

arXiv: 2303.04137

DOI: 10.1177/02783649241273668

Summary

Models robot visuomotor policy as a denoising diffusion process over action sequences, enabling multimodal and high-dimensional action distributions that outperform regression-based imitation learning methods on dexterous manipulation benchmarks.

Abstract

This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process. We benchmark Diffusion Policy across 15 different tasks from 4 different robot manipulation benchmarks and find that it consistently outperforms existing state-of-the-art robot learning methods with an average improvement of 46.9%. Diffusion Policy learns the gradient of the action-distribution score function and iteratively optimizes with respect to this gradient field during inference via a series of stochastic Langevin dynamics steps. We find that the diffusion formulation yields powerful advantages when used for robot policies, including gracefully handling multimodal action distributions, being suitable for high-dimensional action spaces, and exhibiting impressive training stability. To fully unlock the potential of diffusion models for visuomotor policy learning on physical robots, this paper presents a set of key technical contributions including the incorporation of receding horizon control, visual conditioning, and the time-series diffusion transformer. We hope this work will help motivate a new generation of policy learning techniques that are able to leverage the powerful generative modeling capabilities of diffusion models. Code, data, and training details are available (diffusion-policy.cs.columbia.edu).

Links

Primary

Paper arxiv.org

Standard

arXiv Abstract 2303.04137 arXiv PDF 2303.04137 arXiv HTML 2303.04137 DOI 10.1177/02783649241273668

Alternate

journals.sagepub.com journals.sagepub.com dl.acm.org dl.acm.org diffusion-policy.cs.columbia.edu PDF diffusion-policy.cs.columbia.edu diffusion-policy.cs.columbia.edu diffusion-policy.cs.columbia.edu GitHub: real-stanford/diffusion_policy github.com

Tags

• Diffusion policy

• Diffusion models

• Action diffusion

• Machine learning

• Imitation learning

• Robot learning

• Visuomotor policy

• Transformer