Efficient Task-specific Conditional Diffusion Policies: Shortcut Model Acceleration and SO(3) Optimization

Haiyong Yu, Yanqiong Jin, Yonghao He, Wei Sui; Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops, 2025, pp. 4174-4183

Imitation learning, particularly Diffusion Policies based methods, has recently gained significant traction in embod- ied AI as a powerful approach to action policy generation. These models efficiently generate action policies by learn- ing to predict noise. However, conventional Diffusion Pol- icy methods rely on iterative denoising, leading to ineffi- cient inference and slow response times, which hinder real- time robot control. To address these limitations, we pro- pose a Classifier-Free Shortcut Diffusion Policy (CF-SDP) that integrates classifier-free guidance with shortcut-based acceleration, enabling efficient task-specific action gener- ation while significantly improving inference speed. Fur- thermore, we extend diffusion modeling to the SO(3) man- ifold in shortcut model, defining the forward and reverse processes in its tangent space with an isotropic Gaussian distribution. This ensures stable and accurate rotational estimation, enhancing the effectiveness of diffusion-based control. Our approach achieves nearly 5x acceleration in diffusion inference compared to DDIM-based Diffusion Pol- icy while maintaining task performance. Evaluations both on the RoboTwin simulation platform and real-world sce- narios across various tasks demonstrate the superiority of our method.