Research map / Reinforcement learning / Visual and representation RL
A Practical Guide for Incorporating Symmetry in Diffusion Policy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes practical ways to add SE(3)-symmetry to diffusion policies (relative/delta actions with eye-in-hand perception, C8 equivariant encoders, and Frame Averaging on pretrained ResNet-18) and shows they match or beat fully equivariant EquiDiff on MimicGen.
Problem
Fully equivariant diffusion policies are hard to implement and do not easily combine with pretrained vision encoders and modern policy components, so symmetry benefits in visuomotor diffusion policies remain underused.
Approach
Combine eye-in-hand perception with relative/delta trajectory actions (proved SE(3)-invariant/equivariant), and obtain symmetric features either from a C8 equivariant ResNet-18 or from a pretrained ResNet-18 made equivariant via Frame Averaging, keeping a standard diffusion head.
Key results
- Pretrain + FA with a single eye-in-hand image and relative trajectory reaches 61.4 mean success on 12 MimicGen tasks (Large FOV In-Hand), about 2.5% below voxel-based EquiDiff (63.9).
- Relative trajectory beats absolute trajectory by 5.9% (Large FOV In-Hand) and 7.4% (In-Hand + External) on average.
- Equivariant encoder (Equi Enc) improves over CNN encoder by 9.1% average; Frame Averaging improves pretrained encoder by 4.1% average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace absolute trajectory actions with relative trajectory actions | Diffusion Policy with In-Hand + External observations · MimicGen, 12 tasks | mean success rate: 42.0 -> 49.4; 7.4% improvement | Helped |
| Replace absolute trajectory actions with delta trajectory actions | Diffusion Policy with Large FOV In-Hand observations · MimicGen, 12 tasks | mean success rate: 40.8 -> 37.9; underperforming absolute trajectory by 2.9% | Hurt |
| Replace a standard CNN encoder with an equivariant encoder | Diffusion Policy with relative actions and Large FOV In-Hand observations · MimicGen, 12 tasks | mean success rate: 46.7 -> 55.8; 9.1% average improvement | Helped |
| Add Frame Averaging to the pretrained encoder | Diffusion Policy with relative actions and Large FOV In-Hand observations · MimicGen, 12 tasks | mean success rate: 57.3 -> 61.4; 4.1% average performance improvement | Helped |
| Add Frame Averaging to the pretrained encoder | Diffusion Policy with relative actions and In-Hand + External views · MimicGen, 12 tasks | mean success rate: 6.7% improvement compared with not using Frame Averaging | Helped |
| Replace relative trajectory actions with absolute trajectory actions (ablation) | Full model with Pretrain + FA · MimicGen, four-task ablation | mean success rate: 67.0 -> 55.9; reduces performance by 11.1% | Hurt |
| Replace Pretrain + FA with a non-pretrained CNN (ablation) | Full model with relative trajectory actions · MimicGen, four-task ablation | mean success rate: 67.0 -> 47.4; reduces performance by 19.6% | Hurt |
| Remove gripper pose from the policy input (ablation) | Diffusion Policy · MimicGen, Stack Three D1 | success rate: 72.0 -> 64.7 | Hurt |
| Reduce the equivariant encoder group from C8 to C4 (ablation) | Equivariant vision encoder · MimicGen, Stack Three D1 | success rate: 75.3 -> 68.0 | Hurt |
| Reduce the equivariant encoder group from C8 to C2 (ablation) | Equivariant vision encoder · MimicGen, Stack Three D1 | success rate: 75.3 -> 63.3 | Hurt |
| Replace In-Hand + External observations with Large FOV In-Hand only (ablation) | EquiDiff (Im) · MimicGen, 12 tasks | mean success rate: 53.7 -> 40.9; drops by 12.8% | Hurt |
| Add Pretrain + FA to Diffusion Policy using absolute actions and multi-camera observations | Diffusion Policy · MimicGen, four tasks | mean success rate: 36.2 -> 57.7; 21.5% improvement | Helped |
Surprising findings
- Delta trajectory actions underperform absolute trajectories by 2.9% on average despite having the same invariance guarantee.
- Removing external cameras reduces EquiDiff mean success by 12.8%, showing the external view matters for fully equivariant baselines.
Limitations
- Single eye-in-hand view can lack workspace coverage, causing occlusion and failures on long-horizon tasks such as Coffee Preparation.
- Frame Averaging and equivariant encoders are computationally more expensive than standard Diffusion Policy.
- Not demonstrated on physical robots; evaluated only on MimicGen and Robomimic simulation benchmarks.
Experimental setup
Datasets: MimicGen Robomimic ImageNet-1k
Benchmarks: MimicGen Robomimic
Metrics: success rate
Baselines: Diffusion Policy (CNN Enc, absolute trajectory) EquiDiff (Im) EquiDiff (Vo) DiffPo
Builds on: Diffusion Policy Equivariant Diffusion Policy Frame Averaging DDPM escnn
Closest papers by meaning
- 3D Equivariant Visuomotor Policy Learning via Spherical Projection (NeurIPS 2025 · similarity 0.89)
- ET-SEED: EFFICIENT TRAJECTORY-LEVEL SE(3) EQUIVARIANT DIFFUSION POLICY (ICLR 2025 · similarity 0.87)
- Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies (NeurIPS 2025 · similarity 0.87)
- SiT: Symmetry-invariant Transformers for Generalisation in Reinforcement Learning (ICML 2024 · similarity 0.87)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.87)
- Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.