Research map / Reinforcement learning / RL from feedback
From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models
Hongrui Jia, Chaoya Jiang, Yongrui Heng, Shikun Zhang, Wei Ye
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add diagnostic module (ablation) | DPE · CharXiv RQ (Qwen2.5-VL-7B) | accuracy (%): 36.7 -> 40.91 | Improved |
| add diagnostic module (ablation) | DPE · MathVision (Qwen2.5-VL-7B) | accuracy (%): 25.99 -> 26.51 | Improved |
| add image retrieval and editing tools (ablation) | DPE · CharXiv RQ (Qwen2.5-VL-7B) | accuracy (%): 38.1 -> 40.91 | Improved |
| add image retrieval and editing tools (ablation) | DPE · MathVision (Qwen2.5-VL-7B) | accuracy (%): 26.18 -> 26.51 | Improved |
| use DPE diagnosis-driven 3K samples instead of static Vision-SR1 47K data | Qwen2.5-VL-7B-Instruct · MMMU | accuracy (%): 54.8 -> 56.44 | Improved |
| use DPE diagnosis-driven 3K samples instead of static Vision-SR1 47K data | Qwen2.5-VL-7B-Instruct · MathVista | accuracy (%): 68.8 -> 69.5 | Improved |
| use DPE instead of VisPlay (Iter 3) | Qwen2.5-VL-7B-Instruct · Average over 11 benchmarks | accuracy (%): 58.33 -> 59.29 | Improved |
| apply DPE to stronger backbone | Qwen3-VL-8B-Instruct · Average over 11 benchmarks | accuracy (%): 65.64 -> 68.04 | Improved |
| apply DPE to stronger backbone | Qwen3-VL-8B-Instruct · ChartQA | accuracy (%): 85.08 -> 84.80 | No clear effect |
| use DPE instead of VisPlay for question text diversity (Iter 3) | question generation · text diversity | mean pairwise cosine distance: 0.797 -> 0.850 | Improved |
| use DPE instead of VisPlay for question quality (Iter 3) | question generation · LLM-judged quality score QS | score (1-5): 3.32 -> 4.80 | Improved |
Closest papers by meaning
- Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors (NeurIPS 2026 · similarity 0.89)
- CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning (CVPR 2026 · similarity 0.88)
- Beyond SFT-to-RL: Pre-alignment via Black-box On-policy Distillation for Multimodal RL (NeurIPS 2026 · similarity 0.88)
- Diving into Self-Evolving Training for Multimodal Reasoning (ICML 2025 · similarity 0.88)
- MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning (CVPR 2026 · similarity 0.87)
- Meta-TTRL: A Metacognitive Framework for Self-Improving Test-Time Reinforcement Learning for T2I Generation in Unified Multimodal Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
