Research map / Large language models / Instruction and fine-tuning
Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions
Lu Ma, Hao Liang, Meiyi Qiang, Lexiang Tang, Xiaochen Ma, Zhen Hao Wong, Junbo Niu, Chengyu Shen, Runming He, Yanhao Li, Wentao Zhang, Bin CUI
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| interleave online SFT on hardest questions into RL | GRPO RL on Qwen2.5-Math-7B · overall six benchmarks | accuracy (%): 46.9 -> 52.6 | Improved |
| interleave online SFT on hardest questions vs SFT | Qwen2.5-Math-7B · overall six benchmarks | accuracy (%): 49.0 -> 52.6 | Improved |
| use ReLIFT instead of LUFFY | Qwen2.5-Math-7B · overall | accuracy (%): 50.9 -> 52.6 | Improved |
| use ReLIFT instead of SFT then RL | Qwen2.5-Math-7B · overall | accuracy (%): 50.8 -> 52.6 | Improved |
| add ReLIFT on Qwen2.5-Math-1.5B | RL · overall | accuracy (%): 34.2 -> 36.5 | Improved |
| add ReLIFT on Qwen2.5-7B | RL · overall | accuracy (%): 43.1 -> 45.0 | Improved |
| add ReLIFT on Llama3.1-8B | RL · overall | accuracy (%): 14.6 -> 17.3 | Improved |
| add adaptive scheduling vs fine-tuning after every RL step (ablation) | ReLIFT · overall | accuracy (%): 23.8 -> 52.6 | Improved |
| add adaptive scheduling vs uniform intervals (ablation) | ReLIFT · overall | accuracy (%): 49.1 -> 52.6 | Improved |
| select hardest questions instead of random non-hardest (ablation) | ReLIFT · overall | accuracy (%): 45.5 -> 52.6 | Improved |
| add entropy loss with alpha=1e-4 to fine-tuning (ablation) | ReLIFT · Qwen2.5-Math-7B | accuracy: 52.6 | Improved |
| increase buffer size from 32 to 64 at Q=0 (ablation) | ReLIFT · 5 math benchmarks | average accuracy: 0.411 -> 0.490 | Improved |
Closest papers by meaning
- RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.94)
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.93)
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.92)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.92)
- AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy (ICLR 2026 · similarity 0.92)
- Native Reasoning Models: Training Language Models to Reason on Unverifiable Data (ICLR 2026 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
