Research map / Reinforcement learning / RL from feedback
SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start
Kun Chen, Peng Shi, Haibo Qiu, Zhixiong Zeng, Siqi Yang, Wenji Mao, Lin Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SFT cold start with DPO cold start (ablation) | SPECS + GRPO · 5-benchmark average | accuracy: 47.65 -> 50.02 | Improved |
| use self-distilled preference data instead of Qwen2.5-VL-32B teacher data (ablation) | SPECS · 5-benchmark average (after RL) | accuracy: 46.43 -> 50.02 | Improved |
| use self-distilled data instead of Qwen2.5-VL-72B teacher data (ablation) | SPECS · 5-benchmark average (after RL) | accuracy: 48.98 -> 50.02 | Improved |
| distill chosen responses from GRPO-zero instead of base model (ablation) | SPECS · 5-benchmark average (after RL) | accuracy: 48.79 -> 50.02 | Improved |
| use decoupled (format-only) preference pairs instead of coupled pairs (ablation) | SPECS · 5-benchmark average (after RL) | accuracy: 48.68 -> 50.02 | Improved |
| use decoupled data before RL (cold start only) (ablation) | SPECS · 5-benchmark average (cold start only) | accuracy: 47.67 -> 47.27 | Hurt |
| add LLM verification of chosen responses (ablation) | SPECS · 5-benchmark average | accuracy: 48.13 -> 50.02 | Improved |
| add SFT loss to DPO (hybrid loss) in cold start (ablation) | DPO cold start · ID/OOD generalization factor | GF | Improved |
| DPO instead of SFT cold start for exploration (ablation) | SPECS · Rollout branching factor | RBF: 1.8086 -> 1.8216 | Improved |
| apply full SPECS pipeline over backbone | Qwen2.5-VL-7B · MathVista | accuracy: 63.70 -> 75.90 | Improved |
| apply full SPECS pipeline over backbone | Qwen2.5-VL-7B · MEGA-Bench Core | score: 35.07 -> 39.17 | Improved |
| add DPO cold start before GRPO versus GRPO-zero (ablation) | Qwen2.5-VL-7B · MEGA-Bench Core | score: 37.96 -> 39.17 | Improved |
Closest papers by meaning
- Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning (ICLR 2026 · similarity 0.92)
- Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning (CVPR 2026 · similarity 0.91)
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (ICLR 2026 · similarity 0.89)
- Beyond SFT-to-RL: Pre-alignment via Black-box On-policy Distillation for Multimodal RL (NeurIPS 2026 · similarity 0.89)
- MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning (CVPR 2026 · similarity 0.89)
- Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
