Research map / Reinforcement learning / Visual and representation RL
Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
Aojun Lu, Tao Feng, Hangjie Yuan, Wei Li, Yanan Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train SFT on hard-difficulty samples only (ablation) | Qwen2.5-VL-7B SFT · ImageNet-R | accuracy: 57.32 -> 43.25 | Hurt |
| train SFT on hard-difficulty samples only (ablation) | Qwen2.5-VL-3B SFT · ImageNet-R | accuracy: 60.51 -> 51.17 | Hurt |
| train SFT on medium-difficulty samples only (ablation) | Qwen2.5-VL-7B SFT · ImageNet (ID) | accuracy: 78.26 -> 85.52 | Improved |
| train SFT on easy samples only (ablation) | Qwen2.5-VL-7B SFT · ImageNet-R | accuracy: 57.32 -> 59.37 | Improved |
| filter to easy plus medium samples (SFT-EM) instead of full data | SFT · OOD average (7B) | accuracy: 57.62 -> 62.10 | Improved |
| train on medium samples only (SFT-M) instead of full-data SFT | SFT · OOD average (7B) | accuracy: 57.62 -> 60.41 | Improved |
| DC-SFT variants instead of full-data SFT | SFT · ImageNet (ID) | accuracy: 93.70 -> 91.44 | Hurt |
| SFT-M instead of GRPO | GRPO · OOD average (7B) | accuracy: 59.48 -> 60.41 | Improved |
| SFT-EM instead of GRPO | GRPO · OOD average (7B) | accuracy: 59.48 -> 62.10 | Improved |
| add 5% hard data to SFT-EM data (ablation) | SFT-EM · ImageNet-R | accuracy: -3.74 | Hurt |
| add 5% hard data to SFT-EM data (ablation) | SFT-EM · ImageNet-A | accuracy: -2.51 | Hurt |
| DC-SFT instead of GRPO training | GRPO · training efficiency | speedup: 4.9x on ImageNet, 3.2x on RefCOCO | Improved |
Closest papers by meaning
- What Can RL Bring to VLA Generalization? An Empirical Study (NeurIPS 2025 · similarity 0.92)
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training (ICML 2025 · similarity 0.91)
- Understanding the Effects of RLHF on LLM Generalisation and Diversity (ICLR 2024 · similarity 0.90)
- Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead (ICLR 2026 · similarity 0.90)
- RL makes MLLMs see better than SFT (ICLR 2026 · similarity 0.90)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
