Research map / Large language models / Alignment and preferences
Unified Generation and Self-Verification for Vision-Language Models via Advantage Decoupled Preference Optimization
Xinyu Qiu, Heng Jia, Zhengwen Zeng, Shuheng Shen, Changhua Meng, Yi Yang, Linchao Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace binary verification reward with preference verification reward (ablation) | ADPO · AndroidControl | AUC: +19.4% | Improved |
| replace binary verification reward with preference verification reward (ablation) | ADPO · MathVista | best@8 accuracy: +1.6% | Improved |
| replace binary verification reward with preference verification reward (ablation) | ADPO · MathVista | AP: +1.3% | Improved |
| use decoupled advantage instead of entangled advantage (ablation) | ADPO · AndroidControl | AUC: +34.1% | Improved |
| use decoupled advantage instead of entangled advantage (ablation) | ADPO · GUI agent tasks | best@8 performance: +2.8% | Improved |
| naively sum answer and verification rewards (entangled advantage) (ablation) | ADPO · training | generation quality | Hurt |
| use binary verification reward under class imbalance (ablation) | ADPO · training | verification score discrimination | Hurt |
| set preference margin gamma to 0.1 instead of 0.025 (ablation) | ADPO · ReasonSeg (overall) | ACC: 71.1 -> 73.5 | Improved |
| set preference margin gamma >= 0.2 (ablation) | ADPO · ReasonSeg (overall) | ACC: 73.5 -> 72.3 | Hurt |
| use unified ADPO policy as verifier instead of GRPO-as-judge | GRPO generator · MathVista best-of-8 | accuracy (%) / latency: 60.8 -> 65.0 acc; 5.6 -> 2.6 s | Improved |
| ADPO training vs GRPO on pass@1 | ADPO · MMMU (OOD) | accuracy (%): 48.7 -> 47.7 | No clear effect |
| ADPO training vs GRPO on pass@1 | ADPO · ReasonSeg (overall) | cIoU: 59.5 -> 59.1 | No clear effect |
Closest papers by meaning
- Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards (ICML 2026 · similarity 0.87)
- MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation (NeurIPS 2026 · similarity 0.87)
- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback (ICML 2025 · similarity 0.86)
- One-Way Policy Optimization for Self-Evolving LLMs (ICML 2026 · similarity 0.86)
- TTRV: Test-Time Reinforcement Learning for Vision Language Models (CVPR 2026 · similarity 0.86)
- Learning to Self-Verify Makes Language Models Better Reasoners (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
