Research map / Reinforcement learning / Visual and representation RL
Improving Policy Optimization with Generalist-Specialist Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add specialists trained on subsets after the generalist plateaus, then consolidate their demonstrations with DAPG | PPO on BigFish · BigFish train, 1024 levels | episode reward: 24.6$\pm$0.7 -> 31.1$\pm$0.8 | Helped |
| add specialists trained on subsets after the generalist plateaus, then consolidate their demonstrations with DAPG | PPO on BigFish · BigFish test, 1000 unseen levels | episode reward: 24.3$\pm$1.1 -> 30.0 $\pm$0.5 | Helped |
| add specialist-guided DAPG to PPO | PPO on BossFight · BossFight train, 1024 levels | episode reward: 8.6$\pm$0.2 -> 11.3$\pm$0.2 | Helped |
| add specialist-guided DAPG to PPG | PPG on BigFish · BigFish train, 1024 levels | episode reward: 29.4$\pm$1.1 -> 33.5$\pm$1.3 | Helped |
| add specialist-guided DAPG to PPO | PPO on MT-10 · MT-10 train | success rate (%): 58.4$\pm$10.1 -> 77.5$\pm$2.9 | Helped |
| add specialist-guided DAPG to PPO | PPO on MT-50 · MT-50 train | success rate (%): 31.1$\pm$4.5 -> 43.5$\pm$2.2 | Helped |
| add specialist-guided GAIL to SAC | SAC on PushChair · PushChair train | return (k): -2.97$\pm$2.7 -> -2.78$\pm$2.3 | Helped |
| use behavior cloning instead of DAPG for specialist-to-generalist consolidation (ablation) | GSL on BigFish · BigFish train, 256 levels | episode reward: 24.6$\pm$0.7 -> 25.3$\pm$0.4 | Hurt |
| use behavior cloning instead of DAPG for specialist-to-generalist consolidation (ablation) | GSL on BigFish · BigFish test, 1000 novel levels | episode reward: 24.3$\pm$1.1 -> 22.1$\pm$0.9 | Hurt |
| start specialist training before the generalist reaches its performance plateau (ablation) | GSL specialist training · BigFish and PushChair | sample efficiency | Hurt |
| train specialists from scratch instead of initializing them from the generalist checkpoint (ablation) | GSL specialist training · BigFish and PushChair | sample complexity | Hurt |
| vary the number of environment variations assigned per specialist, using smaller subsets rather than one generalist over all levels (ablation) | PPO on BigFish and StarPilot · Procgen, 256 levels | episode reward | Mixed |
Closest papers by meaning
- Instance-based Generalization in Reinforcement Learning (NeurIPS 2020 · similarity 0.89)
- Quantifying Generalization in Reinforcement Learning (ICML 2019 · similarity 0.89)
- Learning Dynamics and Generalization in Deep Reinforcement Learning (ICML 2022 · similarity 0.88)
- Synthesizing Programmatic Policies that Inductively Generalize (ICLR 2020 · similarity 0.88)
- How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Generalizable Policy Improvement via Reinforcement Sampling (Student Abstract) (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.