Research map / Reinforcement learning / Visual and representation RL
DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DexFlyWheel, a self-improving flywheel that generates dexterous manipulation demonstrations from one seed demo per task by iterating IL (diffusion policy), residual SAC RL, rollouts, and MimicGen-style augmentation.
Problem
Dexterous manipulation data is scarce; teleoperation is labor-intensive and existing simulation generation methods (replay, planning, pure RL, LLM-driven) produce low-diversity or non-human-like data.
Approach
Treat human demonstrations as behavioral priors: a diffusion base policy trained by IL is refined by a frozen-base residual SAC policy to handle novel objects, and its successful rollouts are augmented across objects, environments, and spatial poses, then fed into the next iteration, forming a data flywheel.
Key results
- Policies trained on the final iteration dataset reach 81.9% average SR on T_OEP, vs 31.4% for DexMimicGen (Default) and 45.2% for DexMimicGen (Enhanced).
- Data generation success averages 89.8% vs 63.0% for DexMimicGen; Handover 85.7% vs 14.8%.
- Real-world deployment via digital twin achieves 78.3% on dual-arm lift and 63.3% on handover.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DexFlyWheel instead of DexMimicGen (Enhanced) | Policies trained for dexterous manipulation · T_OEP average across four tasks | success rate: 45.2% -> 81.9% | Helped |
| Use DexFlyWheel instead of DexMimicGen (Enhanced) | Grasp policy · Grasp, T_OEP | success rate: 50.3% -> 90.0% | Helped |
| Use DexFlyWheel instead of DexMimicGen (Enhanced) | Pour policy · Pour, T_OEP | success rate: 44.4% -> 85.8% | Helped |
| Use DexFlyWheel instead of DexMimicGen (Enhanced) | Lift policy · Lift, T_OEP | success rate: 43.7% -> 79.4% | Helped |
| Use DexFlyWheel instead of DexMimicGen (Enhanced) | Handover policy · Handover, T_OEP | success rate: 42.5% -> 72.5% | Helped |
| Use DexFlyWheel rather than DexMimicGen | Demonstration generation · Average across four manipulation tasks | successful demonstration generation rate: 63.0% -> 89.8% | Helped |
| Use DexFlyWheel rather than DexMimicGen | Data generation · Lift, iteration 3; 500 successful trajectories | collection time: 4.4 h -> 2.4 h | Helped |
| Advance the flywheel from iteration 1 to iteration 3 | DexFlyWheel policies · T_OEP average across four tasks | success rate: 16.5% -> 81.9% | Helped |
| Add the residual policy to the base policy (ablation) | DexFlyWheel · Object diversity across tasks | average number of successfully manipulated objects: 8.25 -> 20 | Helped |
| Add the residual policy to the base policy (ablation) | DexFlyWheel · Object generalization test sets T_O(i), four tasks | success rate: 48.5% -> 82.6% | Helped |
| Remove the data augmentation module A_EP (ablation) | DexFlyWheel · Four manipulation tasks | success rate | Hurt |
| Remove both the residual policy and A_EP (ablation) | DexFlyWheel · Four manipulation tasks | success rate | Hurt |
Surprising findings
- DexMimicGen (Enhanced) with 10x more human demos barely improves over default, and Human Demo (Enhanced) scores below DexMimicGen, suggesting replay data is limited by object diversity.
- Residual RL improves object generalization substantially; w/o Res drops average object coverage from 20 to 8.25 objects.
Limitations
- Reward functions for RL are manually designed and rely on reward engineering.
- No tactile feedback in policies or simulation.
- Minor-adjustment observation may not hold for deformable or precision-assembly tasks.
Experimental setup
Datasets: single VR-teleoperated seed demonstration per task OmniGibson objects and environments (80 objects, 12 environments)
Benchmarks: T_OEP multi-factor generalization test set (40 unseen configs) T_O(i) object generalization test sets real-world dual-arm lift and handover
Metrics: success rate data generation success rate time per trajectory object/environment/pose diversity counts
Baselines: Human Demo (Default) Human Demo (Enhanced) DexMimicGen (Default) DexMimicGen (Enhanced)
Builds on: MimicGen DexMimicGen Diffusion Policy Soft Actor-Critic OmniGibson
Closest papers by meaning
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.91)
- Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- DexScale: Automating Data Scaling for Sim2Real Generalizable Robot Control (ICML 2025 · similarity 0.89)
- DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References (ICLR 2025 · similarity 0.88)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.88)
- Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.