Research map / Reinforcement learning / RL from feedback
Balancing the Experts: Unlocking LoRA-MoE for GRPO via Mechanism-Aware Rewards
Changlian Ma, Zizheng Huang, Xiangyu Zeng, Yi Wang, Cheng Liang, Kun Tian, Xinhai Zhao, Limin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add balancing reward RB to RO-GRPO (Smooth) (ablation) | RO-GRPO (Smooth) · GSM8K (E=2) | accuracy (%): 90.75 -> 91.51 | Improved |
| add confidence reward RH to RO-GRPO (Smooth) (ablation) | RO-GRPO (Smooth) · GSM8K (E=2) | accuracy (%): 89.92 -> 91.51 | Improved |
| add balancing reward RB to RO-GRPO (Relative) (ablation) | RO-GRPO (Relative) · GSM8K (E=2) | accuracy (%): 89.01 -> 90.22 | Improved |
| add confidence reward RH to RO-GRPO (Relative) (ablation) | RO-GRPO (Relative) · GSM8K (E=2) | accuracy (%): 90.14 -> 90.22 | No clear effect |
| use correct rather than shuffled routing rewards (ablation) | RO-GRPO (Smooth) · GSM8K (E=2) | accuracy (%): 89.23 -> 91.51 | Improved |
| use correct rather than shuffled routing rewards (ablation) | RO-GRPO (Relative) · GSM8K (E=2) | accuracy (%): 89.28 -> 90.22 | Improved |
| add routing reward (Smooth) to GRPO LoRA-MoE | GRPO LoRA-MoE 4x8 · Geometry3k | accuracy (%): 28.95 -> 40.10 | Improved |
| add routing reward (Relative) to GRPO LoRA-MoE | GRPO LoRA-MoE 4x8 · Geometry3k | accuracy (%): 28.95 -> 40.27 | Improved |
| add routing reward to reduce repetitive-loop failures | GRPO LoRA-MoE 4x8 · Geometry3k | repetitive-loop rate: 7.5% -> 0.17% (Smooth), 0% (Relative) | Improved |
| replace auxiliary routing loss with routing reward (Smooth) | LoRA-MoE with GRPO · GSM8K (E=2) | accuracy (%): 86.73 -> 91.51 | Improved |
| replace auxiliary routing loss with routing reward | LoRA-MoE with GRPO · GSM8K (E=2) | load-balancing MSE: 0.036 -> 0.016 | Improved |
| add auxiliary routing loss to GRPO LoRA-MoE | GRPO LoRA-MoE 2x8 · GSM8K | accuracy (%): 89.39 -> 86.73 | Hurt |
Closest papers by meaning
- MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models (CVPR 2026 · similarity 0.90)
- PSPO: Prompt-Level Prioritization and Experience-Weighted Smoothing for Efficient Policy Optimization (AAAI 2026 · similarity 0.89)
- Step-GRPO: Enhancing Reasoning Quality and Efficiency via Structured PRM-Based Reinforcement Learning (AAAI 2026 · similarity 0.88)
- TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback (NeurIPS 2026 · similarity 0.88)
- $f$-GRPO & Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment (NeurIPS 2026 · similarity 0.87)
- BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
