AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start

Kun Chen, Peng Shi, Haibo Qiu, Zhixiong Zeng, Siqi Yang, Wenji Mao, Lin Ma

ICLR 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace SFT cold start with DPO cold start (ablation)SPECS + GRPO · 5-benchmark averageaccuracy: 47.65 -> 50.02Improved
use self-distilled preference data instead of Qwen2.5-VL-32B teacher data (ablation)SPECS · 5-benchmark average (after RL)accuracy: 46.43 -> 50.02Improved
use self-distilled data instead of Qwen2.5-VL-72B teacher data (ablation)SPECS · 5-benchmark average (after RL)accuracy: 48.98 -> 50.02Improved
distill chosen responses from GRPO-zero instead of base model (ablation)SPECS · 5-benchmark average (after RL)accuracy: 48.79 -> 50.02Improved
use decoupled (format-only) preference pairs instead of coupled pairs (ablation)SPECS · 5-benchmark average (after RL)accuracy: 48.68 -> 50.02Improved
use decoupled data before RL (cold start only) (ablation)SPECS · 5-benchmark average (cold start only)accuracy: 47.67 -> 47.27Hurt
add LLM verification of chosen responses (ablation)SPECS · 5-benchmark averageaccuracy: 48.13 -> 50.02Improved
add SFT loss to DPO (hybrid loss) in cold start (ablation)DPO cold start · ID/OOD generalization factorGFImproved
DPO instead of SFT cold start for exploration (ablation)SPECS · Rollout branching factorRBF: 1.8086 -> 1.8216Improved
apply full SPECS pipeline over backboneQwen2.5-VL-7B · MathVistaaccuracy: 63.70 -> 75.90Improved
apply full SPECS pipeline over backboneQwen2.5-VL-7B · MEGA-Bench Corescore: 35.07 -> 39.17Improved
add DPO cold start before GRPO versus GRPO-zero (ablation)Qwen2.5-VL-7B · MEGA-Bench Corescore: 37.96 -> 39.17Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy