Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning with Euclidean Data Augmentation for State-Based Continuous Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Euclidean data augmentation for state-based continuous control by rotating limb-based kinematic features (instead of joint-based ones) under SO_g(3) yaw rotations, applied on top of DDPG.
Problem
Data augmentation for state-based continuous control has had limited benefit because default joint-configuration state features are largely invariant to Euclidean transformations, so perturbation-based augmentation (noise, scaling) is ineffective and principled symmetry-based augmentation is underex
Approach
Replace joint-based kinematic state features with limb-based features (positions, velocities, orientations of rigid bodies) that are equivariant under rotations about gravity, then augment transitions by random yaw rotations of these features while keeping actions and rewards invariant.
Key results
- On Humanoid_run, standard DDPG stays below 100 episode reward while the method reaches about 150 after 5M steps.
- On Hopper3D_hop, standard DDPG stays below 40 while the method exceeds 200 after 2M steps.
- On Reacher_hard, SEGNN and the method with rho_aug=25% are the most data-efficient, both clearly outperforming other baselines; the method takes about 2 hours per 1M steps versus over 70 hours for SEGNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace joint-based features with limb-based features and add task-tuned Euclidean rotation augmentation | DDPG · Humanoid_run | episode reward: below 100 -> 150 | Helped |
| Replace joint-based features with limb-based features and add task-tuned Euclidean rotation augmentation | DDPG · Hopper3D_hop | episode reward: below 40 -> more than 200 | Helped |
| Use limb-based state representation without augmentation (rho_aug=0%) (ablation) | DDPG · 6 tasks shown in the first 2 rows | data efficiency | Helped |
| Add a positive proportion of Euclidean-rotated transitions (ablation) | Limb-based DDPG · Hopper3D_hop and 3 tasks in the last row | performance | Helped |
| Set rho_aug to 100% (ablation) | Limb-based DDPG · Hopper3D_hop | performance | Hurt |
| Set rho_aug to 100% (ablation) | Limb-based DDPG · Humanoid_run | performance | Helped |
| Add Euclidean rotation augmentation with rho_aug>0% (ablation) | Limb-based DDPG · Cheetah_run | performance | No clear effect |
| Replace limb-based features with joint-based features for Euclidean rotation augmentation (ablation) | DDPG · Humanoid_run and Humanoid_stand | data efficiency | Hurt |
| Add Gaussian noise to the joint-based state representation | DDPG · Most of the 9 compared tasks | performance | Hurt |
| Apply random amplitude scaling to the joint-based state representation | DDPG · Most tasks; Cheetah3D_run and Humanoid_stand are exceptions | data efficiency | Mixed |
| Use an equivariant SEGNN policy and critic | DDPG · Reacher_hard | data efficiency | Helped |
| Use a single-point Euclidean graph containing all state features (ablation) | SEGNN · Cheetah3D_run and Reacher_hard | performance | Hurt |
Surprising findings
- Perturbation-based augmentations GN and RAS often hurt DDPG performance on most tasks.
- Rotating the original joint-based features gives little benefit, since only the torso features change under rotation.
Limitations
- Requires task-specific tuning of the augmentation proportion rho_aug.
- Requires knowledge and annotation of strict Euclidean symmetries (gravity direction), which may not hold under noisy external forces such as wind.
- Restricted to locomotion-style MuJoCo/DMControl tasks; other continuous control domains may lack Euclidean symmetry.
Experimental setup
Datasets: DeepMind Control Suite tasks (Cheetah, Hopper, Walker, Quadruped, Reacher, Humanoid) and 3D variants
Benchmarks: DMControl Cheetah_run Cheetah3D_run Hopper_hop Hopper3D_hop Walker_run Walker3D_run Quadruped_run Reacher_hard Humanoid_run Humanoid_stand
Metrics: episode reward data efficiency (learning curves) wall-clock run time
Baselines: DDPG SAC DDPG + GN DDPG + RAS SEGNN joint-based data augmentation
Builds on: DDPG MDP symmetries RAD SEGNN
Closest papers by meaning
- Understanding when Dynamics-Invariant Data Augmentations Benefit Model-free Reinforcement Learning Updates (ICLR 2024 · similarity 0.89)
- Revisiting Data Augmentation in Deep Reinforcement Learning (ICLR 2024 · similarity 0.88)
- CCIL: Continuity-Based Data Augmentation for Corrective Imitation Learning (ICLR 2024 · similarity 0.87)
- Zero Shot Generalization of Vision-Based RL Without Data Augmentation (ICML 2025 · similarity 0.86)
- Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning (ICML 2025 · similarity 0.85)
- Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.