Research map / Image generation / Fast diffusion sampling
Let's Verify and Reinforce Image Generation Step by Step
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Investigates chain-of-thought-style test-time verification (ORM, PRM, and a new PARM) and DPO alignment for autoregressive text-to-image generation on Show-o, reaching 0.77 on GenEval.
Problem
Whether CoT-style reasoning strategies (test-time verification with reward models and preference alignment) transfer to step-wise autoregressive image generation, and how to build reward models suited to blurry intermediate images.
Approach
Adapts ORM/PRM best-of-N verification and iterative DPO to autoregressive image generation, then proposes PARM, a step-adaptive reward model with clarity judgment, potential assessment, and best-of-N' selection, and combines it with DPO.
Key results
- Show-o baseline 0.53 on GenEval improves to 0.63 with fine-tuned ORM best-of-20 and to 0.65 with iterative DPO.
- PARM combined with iterative DPO in training and test time reaches 0.77 overall on GenEval, +24% over baseline.
- Best configuration surpasses Stable Diffusion 3 (d=24) at 0.62 by +15%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use fine-tuned ORM for best-of-20 selection instead of no verifier | Show-o · GenEval | Overall: 0.53 -> 0.63 | Helped |
| Use fine-tuned PRM for best-of-20 selection instead of no verifier | Show-o · GenEval | Overall: 0.53 -> 0.55 | Helped |
| Fine-tune the ORM using ranking data rather than use it zero-shot (ablation) | Show-o with ORM verification · GenEval | Overall: 0.61 -> 0.63 | Helped |
| Fine-tune the PRM using ranking data rather than use it zero-shot (ablation) | Show-o with PRM verification · GenEval | Overall: 0.53 -> 0.55 | Helped |
| Add one-stage DPO alignment | Show-o · GenEval | Overall: 0.53 -> 0.62 | Helped |
| Add iterative DPO training after initial DPO alignment (ablation) | Show-o with DPO · GenEval | Overall: 0.62 -> 0.65 | Helped |
| Add fine-tuned ORM reward guidance to iterative DPO (ablation) | Show-o with iterative DPO · GenEval | Overall: 0.65 -> 0.67 | Helped |
| Apply fine-tuned ORM verification after iterative DPO (ablation) | Show-o with iterative DPO · GenEval | Overall: 0.65 -> 0.72 | Helped |
| Add fine-tuned ORM reward guidance and verification to iterative DPO (ablation) | Show-o with iterative DPO · GenEval | Overall: 0.65 -> 0.75 | Helped |
| Use PARM rather than fine-tuned ORM for verification | Show-o · GenEval | Overall: 0.63 -> 0.67 | Helped |
| Add PARM reward guidance during iterative DPO | Show-o with iterative DPO · GenEval | Overall: 0.65 -> 0.69 | Helped |
| Apply PARM verification after iterative DPO | Show-o with iterative DPO · GenEval | Overall: 0.65 -> 0.74 | Helped |
Surprising findings
- PRM provides only marginal gains (+2%) over the baseline because early-stage images are too blurry and later-stage images are too similar to discriminate.
- Iterative DPO alone slightly outperforms the best test-time verifier (fine-tuned ORM), and combining both gives further gains.
Limitations
- Evaluated only on GenEval, a single text-to-image benchmark focused on object-level composition.
- Relies on GPT-4-generated object lists and Show-o-synthesized labels, limiting data diversity.
- Test-time verification requires sampling N=20 candidates, adding inference cost.
Experimental setup
Datasets: GenEval prompt templates (13K synthetic prompts) self-synthesized Show-o image ranking data
Benchmarks: GenEval
Metrics: GenEval overall score best-of-N accuracy per sub-task
Baselines: Show-o PixArt-α SD v2.1 DALL-E 2 SDXL SD 3 LlamaGen Chameleon LWM SEED-X zero-shot ORM zero-shot PRM
Builds on: Direct Preference Optimization Show-o LLaVA-OneVision iterative DPO GenEval
Closest papers by meaning
- Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.91)
- T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT (NeurIPS 2025 · similarity 0.90)
- GoT-R1: Unleashing Reasoning Capability of Autoregressive Visual Generation with Reinforcement Learning (ICLR 2026 · similarity 0.89)
- Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.87)
- Let VLMs Grade Their Own Thoughts: A Self-Quantification Approach to Reasoning-Aware Reward Modeling (CVPR 2026 · similarity 0.86)
- MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.