Research map / Reinforcement learning / Visual and representation RL
DittoGym: Learning to Control Soft Shape-Shifting Robots
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes coarse-to-fine policy (CFP) with a fully-convolutional SAC policy over a 2D muscle field in an MPM simulator, and the DittoGym benchmark of eight tasks for reconfigurable soft robots.
Problem
Controlling reconfigurable soft robots that change morphology within their lifetime requires very high-dimensional actions for fine-grained morphology change, which makes RL exploration fail; no benchmark or simulator existed for such robots.
Approach
Train a coarse-resolution muscle-field policy first to get meaningful large-scale morphology exploration, then freeze it and learn a residual fine-resolution policy blended via a sigmoid gating mask, within a fully-convolutional SAC policy on an MPM simulator with von Mises plasticity.
Key results
- CFP outperforms GNN and Transformer modular baselines and fine-from-scratch and coarse ablations across all eight DittoGym tasks in sample efficiency and final reward.
- On the SLOT task, CFP reaches 15.4% success vs 5.9% for Transformer-based (dense-segment) and 0% for GNN-based (dense-segment).
- Expert policies at higher action resolution (16x16x2) achieve higher maximum reward than 8x8x2 and 4x4x2 across the eight environments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CFP instead of the Transformer-based dense-segment baseline | Reconfigurable soft robot control · RUN | average time per episode (s): 104 -> 54 | Helped |
| Use CFP instead of the GNN-based dense-segment baseline | Reconfigurable soft robot control · RUN | average time per episode (s): 83 -> 54 | Helped |
| Use CFP instead of the Transformer-based dense-segment baseline | Reconfigurable soft robot control · RUN | used RAM (MB): 6895 -> 4053 | Helped |
| Use CFP instead of the GNN-based dense-segment baseline | Reconfigurable soft robot control · RUN | used RAM (MB): 4199 -> 4053 | Helped |
| Use CFP instead of the Transformer-based dense-segment baseline | Reconfigurable soft robot control · SLOT | success rate (%): 5.9 -> 15.4 | Helped |
| Use CFP instead of the GNN-based dense-segment baseline | Reconfigurable soft robot control · SLOT | success rate (%): 0 -> 15.4 | Helped |
| Add coarse-to-fine training to a fine-resolution policy (ablation) | CFP · DittoGym tasks | episode reward | Helped |
| Add fine-resolution residual training to a coarse policy (ablation) | CFP · DittoGym tasks | episode reward | Helped |
| Increase action resolution from coarse to fine (ablation) | Expert policies · DittoGym (8 environments) | maximum episode reward | Helped |
| Randomly zero out 20% of the action output (ablation) | CFP expert policy · DittoGym tasks | performance: <10%; DIG: 30% | Hurt |
| Randomly zero out 20% of the observation input (ablation) | CFP expert policy · DittoGym tasks | performance | Hurt |
Surprising findings
- Fine-from-scratch policy fails to match the star shape despite having sufficient control resolution, attributed to exploration difficulty in high-dimensional action space.
- Transformer-based baseline matches CFP on CATCH but is highly oscillatory and unstable.
Limitations
- Only 3 seeds per configuration and modest absolute success rates (15.4% on SLOT).
- Simulation is 2D and relies on a simplified MPM model of the magnetic slime robot, not validated on hardware in this work.
Experimental setup
Datasets: DittoGym
Benchmarks: DittoGym
Metrics: episode reward sample efficiency success rate (SLOT) running time per episode RAM usage
Baselines: Modular Policy (GNN-based) Transformer-based modular policy Coarse policy (ablation) Fine-from-scratch policy (ablation)
Builds on: Soft Actor-Critic Material Point Method EvoGym Modular Policy
Closest papers by meaning
- SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments (ICLR 2023 · similarity 0.90)
- Knowledge Diversion for Efficient Morphology Control and Policy Transfer (ICML 2026 · similarity 0.89)
- MorphVAE: Advancing Morphological Design of Voxel-Based Soft Robots with Variational Autoencoders (AAAI 2024 · similarity 0.87)
- Convergent Functions, Divergent Forms (NeurIPS 2025 · similarity 0.87)
- Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control (ICML 2024 · similarity 0.87)
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.