Research map / Reinforcement learning / RL from feedback
EchoRL: Reinforcement Learning via Rollout Echoing
Jinhe Bi, Aniri, Minglai Yang, Xingcheng Zhou, Wenke Huang, Sikuan Yan, Yujun Wang, Zixuan Cao, Michael Färber, Xun Xiao, Volker Tresp, Yunpu Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EchoRL auxiliary loss to GRPO | GRPO, Qwen2.5-Math-7B · math ID average | avg accuracy: 44.5 -> 47.0 | Improved |
| add EchoRL to DAPO | DAPO, Qwen2.5-Math-7B · OOD average | avg accuracy: 57.6 -> 60.9 | Improved |
| add EchoRL to LUFFY | LUFFY, Qwen2.5-Math-7B · math ID average | avg accuracy: 50.1 -> 51.9 | Improved |
| add EchoRL to UFT | UFT, Qwen2.5-Math-7B · math ID average | avg accuracy: 45.7 -> 47.6 | Improved |
| add EchoRL to GRPO | GRPO, Qwen2.5-Math-7B · AIME24 | avg@32: 25.8 -> 24.9 | Hurt |
| add EchoRL to LUFFY | LUFFY, Qwen2.5-Math-7B · Olympiad | accuracy: 57.2 -> 55.1 | Hurt |
| add EchoRL to GRPO | GRPO · training entropy, Qwen2.5-Math-7B | step-level entropy | Improved |
| set auxiliary loss weight lambda to 0.1 or larger instead of 0.001 (ablation) | EchoRL · Qwen2.5-Math-7B | accuracy | Hurt |
| supervise EchoClip (prefix to max-entropy step) instead of full rollout or low-entropy clip (ablation) | EchoRL · Appendix C ablation | accuracy | Improved |
| activate EchoRL late in training instead of full training (ablation) | EchoRL · DAPO vs GRPO | accuracy | Mixed |
| remove reasoning steps by highest entropy first vs lowest-first or random (ablation) | OpenR1-Math 45k trajectories · step removal analysis | accuracy | Hurt |
| add EchoRL to GRPO | GRPO · mean response length | response length | No clear effect |
Closest papers by meaning
- ROLLVERIFY: BRIDGING EFFICIENCY AND ACCURACY IN LONG-TAIL ROLLOUT REINFORCEMENT LEARNING (NeurIPS 2026 · similarity 0.91)
- TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training (NeurIPS 2026 · similarity 0.89)
- Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.89)
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.89)
- Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor’s Internal States (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
