Research map / Reinforcement learning / RL from feedback
QuRL: Low-Precision Reinforcement Learning for Efficient Reasoning
Yuhang Li, Reena Elangovan, Xin Dong, Priyadarshini Panda, Brucek Khailany
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use decoupled/ACR QuRL instead of plain quantized RL (INT8) | RL INT8 · GSM8K (PPO) | accuracy: 48.78 -> 53.55 | Improved |
| use QuRL ACR instead of FlashRL TIS (INT8) | FlashRL INT8 · GSM8K (PPO) | accuracy: 51.40 -> 53.55 | Improved |
| use QuRL ACR instead of FlashRL TIS (FP8) | FlashRL FP8 · GSM8K (PPO) | accuracy: 53.60 -> 54.28 | Improved |
| use quantized FP8 rollout without decoupled correction | RL FP8 · GSM8K (PPO) | accuracy: 55.35 -> 0.0 | Hurt |
| quantize rollout to INT8 with plain RL | RL BF16 · AIME 2024 (DAPO) | Avg@32: 31.67 -> 0.001 | Hurt |
| add UAQ (invariant scaling s=1.5) to QuRL INT8 (ablation) | QuRL w/o UAQ INT8 · AIME 2024 (DAPO) | Avg@32: 30.63 -> 31.25 | Improved |
| add UAQ to QuRL FP8 (ablation) | QuRL w/o UAQ FP8 · AIME 2024 (DAPO) | Avg@32: 33.12 -> 33.27 | No clear effect |
| add UAQ to QuRL INT8 (ablation) | QuRL w/o UAQ INT8 · DeepScaleR 5-task average | Avg@32 average: 54.79 -> 55.48 | Improved |
| add QuRL with UAQ to INT8 RL | RL INT8 · DeepScaleR 5-task average | Avg@32 average: 52.31 -> 55.48 | Improved |
| use QuRL w/ UAQ instead of FlashRL (INT8) | FlashRL INT8 · DeepScaleR 5-task average | Avg@32 average: 53.80 -> 55.48 | Improved |
| raise UAQ scale from 1.5 to 2 (ablation) | QuRL INT8 · AIME 2024 (DAPO INT8) | Avg@32: 31.25 -> 29.15 | Hurt |
| increase learning rate instead of UAQ scale (ablation) | QuRL INT8 s=1 · AIME 2024 (DAPO INT8) | Avg@32: 30.63 -> 26.66 | Hurt |
Closest papers by meaning
- Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.89)
- What are Key Factors for Updates in RL for LLM Reasoning? (NeurIPS 2026 · similarity 0.89)
- Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.89)
- AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning (NeurIPS 2025 · similarity 0.88)
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.88)
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
