Research map / Large language models / Instruction and fine-tuning
Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMs
Sagnik Mukherjee, Lifan Yuan, Pavan Jayasinha, Dilek Hakkani-Tür, Hao Peng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace AdamW with SGD | Qwen3-8B GRPO · Math mean (3K) | mean accuracy: 55.6 -> 55.8 | No clear effect |
| replace AdamW with SGD | Qwen3-1.7B GRPO · Math mean (3K) | mean accuracy: 46.0 -> 48.6 | Improved |
| replace AdamW with SGD | Qwen3-1.7B GRPO · Math mean (8K) | mean accuracy: 58.2 -> 56.8 | Hurt |
| replace AdamW with SGD | Qwen3-1.7B GRPO · HumanEval/MBPP pass@1 (4K) | pass@1: +8.7 average pass@1 | Improved |
| replace AdamW with SGD | Qwen3-1.7B GRPO · MBPP+ pass@1 (4K) | pass@1: 46.8 -> 59.0 | Improved |
| add momentum to SGD (ablation) | Qwen3-8B GRPO · Math mean (3K) | mean accuracy: 55.8 -> 47.1 | Hurt |
| add momentum to SGD (ablation) | Qwen3-1.7B GRPO · Math mean (3K) | mean accuracy: 48.6 -> 48.9 | Improved |
| add adaptive learning rate (RMSProp) to SGD (ablation) | Qwen3-8B GRPO · Math mean (8K) | mean accuracy: 70.0 -> 71.1 | Improved |
| add adaptive learning rate (RMSProp) to SGD (ablation) | Llama-3.1-8B GRPO · Math mean (3K) | mean accuracy: 29.4 -> 29.5 | No clear effect |
| replace AdamW with SGD | Qwen3-8B PPO · Math mean | mean accuracy: 54.0 -> 56.7 | Improved |
| replace AdamW with SGD | Qwen3-1.7B GRPO on RLVE · Math mean (16K) | mean accuracy: 64.5 -> 63.0 | No clear effect |
| replace AdamW with SGD | Qwen3-8B math GRPO · parameter update sparsity (%) | sparsity: 91.30 -> 99.99 | Improved |
Closest papers by meaning
- Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps (NeurIPS 2024 · similarity 0.90)
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.90)
- You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories (NeurIPS 2026 · similarity 0.89)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.89)
- The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning (NeurIPS 2025 · similarity 0.89)
- Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMs (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
