Research map / Reinforcement learning / Policy gradient and actor-critic
ScaleMoE: Mixture-of-Experts for Scalable Continuous Control in Actor-Critic Reinforcement Learning
Yi Ma, Chenjun Xiao, Hongyao Tang, Yaodong Yang, Jinyi Liu, Jing Liang, Jiye Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale from 1 to 4 experts (feature-level gating) (ablation) | ScaleMoE-Simba · DMC dog-run | IQM return: +32% | Improved |
| scale to 8 experts (ablation) | ScaleMoE-Simba · DMC humanoid-run | IQM return: up to +60% | Improved |
| use 8 small experts (width 512) instead of single BRC critic width 4096 | BRC · DMC DOGS | IQM return: 835 -> 849 | Improved |
| use 8 small experts instead of width 4096 monolithic critic (15% parameters) | BRC · DMC HUMANOIDS | IQM return: 709 -> 730 | Improved |
| use 16 experts with full activation instead of width 4096 BRC | BRC · HumanoidBench | IQM normalized return: 0.49 -> 0.62 | Improved |
| switch from feature-level to output-level gating in single-task (ablation) | ScaleMoE · DMC single-task | return | Hurt |
| switch gating level in multi-task (ablation) | ScaleMoE · multi-task DMC | return | No clear effect |
| use top-1 expert activation (K=1) (ablation) | ScaleMoE-Output · DMC single and multi-task | normalized return | Hurt |
| increase total experts N from 2 to 4 to 8 (K=2) (ablation) | ScaleMoE-Simba · DMC single-task | mean final score: 675 -> 697 -> 698 | Mixed |
| replace top-K gating MoE with SoftMoE (ablation) | BRC · MetaWorld 50 tasks | IQM success rate | Hurt |
| replace naive BRC ensemble with ScaleMoE (ablation) | BRC · MetaWorld 50 tasks | IQM success rate | Improved |
| vary auxiliary regularization weight lambda from 0.005 to 1 (ablation) | ScaleMoE · MetaWorld 50 tasks | IQM success rate | No clear effect |
Closest papers by meaning
- Mixtures of Experts Unlock Parameter Scaling for Deep RL (ICML 2024 · similarity 0.91)
- Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer (ICML 2025 · similarity 0.88)
- Offline Actor-Critic Reinforcement Learning Scales to Large Models (ICML 2024 · similarity 0.88)
- Bigger, Regularized, Optimistic: scaling for compute and sample efficient continuous control (NeurIPS 2024 · similarity 0.87)
- Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
