Research map / Reinforcement learning / Visual and representation RL
Flow-based Domain Randomization for Learning and Sequencing Robotic Skills
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GoFlow, which learns a normalizing-flow sampling distribution over domain randomization parameters via entropy-regularized reward maximization with actor-critic PPO, and uses the flow and privileged value function to define belief-space preconditions for multi-step gear assembly planning.
Problem
Choosing hand-specified domain randomization distributions is delicate: too broad causes unsolvable environments and poor convergence, too narrow hurts sim-to-real robustness. The paper also asks how learned sampling distributions can indicate when a policy will succeed under partial observability f
Approach
Replace parametric (Gaussian, beta, uniform) learned sampling distributions with a neural spline normalizing flow, optimized jointly with the policy via reward-weighted entropy plus a self-paced KL term, with importance sampling from a uniform distribution to avoid collapse. The flow and a privileged value function are then reused as out-of-distribution detectors / belief-space preconditions for a
Key results
- GoFlow achieves the highest coverage in simulation, e.g. 0.138 on Cartpole vs 0.103 for LSDR and 0.275 on Quadruped vs 0.086 for NoDR.
- On real-world gear insertion over 10 trials, GoFlow succeeded 9/10 versus 6/10 for FullDR, 5/10 for ADR, DORAEMON and LSDR, and 3/10 for NoDR.
- Statistical tests (one-tailed z test) mark GoFlow as the best or not significantly worse than the best in all six simulated domains reported in its table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GoFlow instead of LSDR | Cartpole · Cartpole | final reward: 0.103$\pm$0.013 -> 0.138$\pm$0.005 | Helped |
| Use GoFlow instead of LSDR | Ant · Ant | final reward: 0.000$\pm$0.000 -> 0.056$\pm$0.033 | Helped |
| Use GoFlow instead of DORAEMON | Quadcopter · Quadcopter | final reward: 0.007$\pm$0.003 -> 0.010$\pm$0.006 | Helped |
| Use GoFlow instead of NoDR | Quadruped · Quadruped | final reward: 0.086$\pm$0.008 -> 0.275$\pm$0.019 | Helped |
| Use GoFlow instead of FullDR | Humanoid · Humanoid | final reward: 0.195$\pm$0.044 -> 0.274$\pm$0.076 | Helped |
| Use GoFlow instead of NoDR | Gears · Gears | final reward: 0.008$\pm$0.003 -> 0.035$\pm$0.019 | Helped |
| Use GoFlow instead of FullDR | Pink gear insertion · Pink gear insertion, real-world | success rate: 6/10 -> 9/10 | Helped |
| Use a normalizing-flow learned sampling distribution rather than fixed or other learned domain-randomization distributions | GoFlow · Simulated environments | coverage | Helped |
Limitations
- Training instability of the normalizing flow leads to higher variance across seeds.
- Planner requires manually tuned, environment-specific thresholds (eta, epsilon, J_T).
- Real-world evaluation is limited to a single gear insertion task with 10 trials per method.
Experimental setup
Datasets: IsaacLab environments (Cartpole, Ant, Quadcopter, Quadruped, Humanoid) custom gear insertion simulation
Benchmarks: IsaacLab Cartpole IsaacLab Ant IsaacLab Quadcopter IsaacLab Quadruped IsaacLab Humanoid gear insertion (simulated and real Franka)
Metrics: coverage mean final reward CVaR real-world success rate
Baselines: NoDR FullDR ADR LSDR DORAEMON
Builds on: PPO ADR LSDR DORAEMON neural spline flows
Closest papers by meaning
- Domain Randomization via Entropy Maximization (ICLR 2024 · similarity 0.91)
- Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations (NeurIPS 2025 · similarity 0.87)
- SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer (NeurIPS 2025 · similarity 0.86)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.86)
- Oracle-Efficient Reinforcement Learning for Max Value Ensembles (NeurIPS 2024 · similarity 0.86)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.