Research map / Reinforcement learning / RL from feedback
Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shuffle
Linghao Zhu, Yiran Guan, Dingkang Liang, Jianzhong Ju, Zhenbo Luo, Bin Qin, Jian Luan, Yuliang Liu, Xiang Bai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Pairwise Trajectory Sampling (PTS) to GRPO (ablation) | GRPO on Qwen2.5-VL-3B · Geo3K test | accuracy: 42.64 -> 46.21 | Improved |
| add Advantage-based Batch Shuffle (ABS) on top of PTS (ablation) | Shuffle-R1 on Qwen2.5-VL-3B · Geo3K test | accuracy: 46.21 -> 47.88 | Improved |
| apply ABS to uniformly random trajectory pairs without PTS (ablation) | GRPO on Qwen2.5-VL-3B · Geo3K test | accuracy: 42.64 -> 46.82 | Improved |
| select only highest-advantage trajectories (one-way positive) (ablation) | PTS · Geo3K test | accuracy: 42.64 -> 41.26 | Hurt |
| select only lowest-advantage trajectories (one-way negative) (ablation) | PTS · Geo3K test | accuracy: 42.64 -> 23.36 | Hurt |
| use max-min contrastive pairing instead of random trajectory selection (ablation) | PTS · Geo3K test | accuracy: 43.53 -> 46.21 | Improved |
| use advantage-weighted shuffling instead of uniform shuffle (ablation) | ABS · Geo3K test | accuracy: 46.05 -> 47.88 | Improved |
| static reorder of batch without sub-sampling (ablation) | PTS-only · Geo3K test | accuracy: 46.21 -> 46.28 | No clear effect |
| lower PTS sampling ratio alpha from 0.5 to 0.25 (ablation) | Shuffle-R1 · Geo3K test | accuracy: 47.88 -> 45.75 | Hurt |
| raise PTS sampling ratio alpha from 0.5 to 0.75 (ablation) | Shuffle-R1 · Geo3K test | accuracy: 47.88 -> 47.41 | Hurt |
| increase ABS shuffle rounds S from 8 to 16 (ablation) | Shuffle-R1 · Geo3K test | accuracy: 47.88 -> 47.23 | Hurt |
| replace ABS with prioritized experience replay (ablation) | Shuffle-R1 · Geo3K training/validation accuracy | accuracy | Hurt |
Closest papers by meaning
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.91)
- Stable and Efficient Single-Rollout RL for Multimodal Reasoning (CVPR 2026 · similarity 0.91)
- Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning (ICLR 2026 · similarity 0.90)
- Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling (NeurIPS 2025 · similarity 0.90)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.90)
- Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
