AI Paper Atlas

Research map / Continual learning / Continual fine-tuning of LLMs

Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective

Zhihao Zhang, Qiaole Dong, Qi Zhang, Enyu Zhou, Jun Zhao, Zhiheng Xi, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Mingqi Wu, Yanwei Fu, Tao Ji and 3 more

ICLR 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Continual pretraining of language models

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train SFT on GRPO-rollout reasoning data instead of human/GPT-4o data (ablation)SFT · RefCOCO val, Qwen2.5-VL-7Baccuracy: 52.5 -> 81.4Improved
train SFT on GRPO-rollout reasoning data instead of GPT-4o reasoning data (ablation)SFT · POPE, Qwen2.5-VL-3Baccuracy: 69.4 -> 85.4Improved
train SFT on GRPO-rollout data instead of GPT-4o data (ablation)SFT · Jigsaw 3x3, Qwen2.5-VL-3Baccuracy: 70.0 -> 70.0No clear effect
use SFT with non-reasoning answers instead of RFTSFT-Non-Rea · MME sum, Qwen2.5-VL-7Bscore: 2333 -> 479Hurt
use RFT (GRPO) instead of SFTRFT · prior benchmarks, Qwen2.5-VL-3B/7Baccuracy: RefCOCO val 88.8 -> 88.4 (3B), 90.0 -> 89.4 (7B)Improved
add reasoning trajectories to SFT data (vs non-reasoning) (ablation)SFT · prior knowledge, Qwen2.5-VLLBK interference and forgetting: RefCOCO val 6.1 -> 74.2 (3B)Improved
use SFT data of lower perplexity under base model (GRPO rollout vs 4o rollout) (ablation)SFT · prior knowledgeforgetting: MME 7B 2084 -> 2207Improved
self-generate CoTs from one epoch of RFT then SFT (ablation)SFT-Rea-Self-Generated · Jigsaw 3x3, Qwen2.5-VL-3Baccuracy: 70.0 -> 84.0Improved
train RFT/SFT on math reasoning with GRPO rollouts instead of Non-Rea (ablation)SFT · GSM8k, Qwen2.5-7B-Instructaccuracy: 21.8 -> 90.3Improved
raise SFT learning rate to 2e-5 with Rea-4o-Rollout (ablation)SFT · RefCOCO val, Qwen2.5-VL-3Baccuracy: 88.8 -> 0.1Hurt
scale RFT to tens of thousands of stepsRFT · Jigsaw 3x3accuracy: 0.0 -> 66 (3B), 0.0 -> 75 (7B)Improved
periodic restart of GRPO (reset optimizer and reference policy every 2 epochs) (ablation)GRPO · Jigsaw training stabilityKL divergenceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy