Research map / Reinforcement learning / RL from feedback
QeRL: Beyond Efficiency - Quantization-enhanced Reinforcement Learning for LLMs
Wei Huang, Yi Ge, Shuai Yang, Yicheng Xiao, Huizi Mao, Yujun Lin, Hanrong Ye, Sifei Liu, Ka Chun Cheung, Hongxu Yin, Yao Lu, Xiaojuan Qi and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Adaptive Quantization Noise on top of NVFP4 + LoRA (ablation) | QeRL · GSM8K, GRPO | accuracy: 88.5 -> 90.8 (7B); 83.3 -> 83.7 (3B) | Improved |
| add AQN on top of NVFP4 + LoRA (ablation) | QeRL · BigMath DAPO, average of 4 benchmarks | average accuracy: 7B: 37.0 -> 36.4; 14B: 40.5 -> 42.0; 32B: 41.4 -> 45.6 | Mixed |
| quantize policy weights to NVFP4 with LoRA instead of BF16 LoRA (ablation) | RL with LoRA · GSM8K, GRPO | accuracy: 88.1 -> 88.5 (7B); 76.1 -> 83.3 (3B) | Improved |
| use NVFP4 instead of NF4 quantization for RL with LoRA (ablation) | QLoRA · GSM8K, GRPO | accuracy: 85.0 -> 88.5 (7B) | Improved |
| use NVFP4 instead of MXFP4 (ablation) | QeRL · GSM8K, GRPO | accuracy: 86.4 -> 88.5 (7B); 73.4 -> 83.3 (3B) | Improved |
| replace NF4 QLoRA with NVFP4 QeRL for speed | RL training · end-to-end GRPO training | speedup vs LoRA: x0.8 -> x1.5 (7B, batch 2) | Improved |
| exponential decay noise schedule instead of linear, cosine or logarithmic (ablation) | AQN · GSM8K, Qwen2.5-3B | reward | Improved |
| vary LoRA rank among 16, 32, 64, 128 (ablation) | QeRL · GSM8K, Qwen2.5-3B | reward growth | No clear effect |
| raise learning rate to 3e-5 (ablation) | QeRL vs BF16 LoRA · Qwen2.5-7B-Instruct | training stability | Mixed |
| compare quantized model with no RL training | NVFP4 post-training quantization · MATH500 etc., 7B | average accuracy: 28.9 -> 25.7 | Hurt |
Closest papers by meaning
- ShiQ: Bringing back Bellman to LLMs (NeurIPS 2025 · similarity 0.91)
- Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.90)
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.89)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.89)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.89)
- Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
