Research map / Image generation / Text-to-image diffusion

Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation

NeurIPS 2025 · Image generation › Text-to-image diffusion · approach: Preference optimization for diffusion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mask-GRPO, the first GRPO-based RL method for masked generative models, redefining the transition probability over unmasking steps, and applies it to Show-o (1.3B) for text-to-image generation with CLIP rewards.

Problem

Applying reinforcement learning to masked generative models (MGMs) for text-to-image generation is unexplored, mainly because the transition probability for parallel multi-token unmasking is not well defined.

Approach

Model the iterative unmasking process as a multi-step MDP and define the transition probability over newly unmasked tokens (two candidate definitions, p_theta1 and p_theta2) so GRPO can be applied; add KL removal, reduced unmasking steps during training, and reward-std-based sample filtering.

Key results

• Show-o+Mask-GRPO reaches 0.73 GenEval overall vs 0.53 for base Show-o, outperforming listed diffusion, AR and MGM baselines.
• MSCOCO-30K FID improves from 9.24 (Show-o) to 8.32 with Mask-GRPO.
• Removing KL raises GenEval to 0.73 vs 0.62 with KL; unmasking reduction gives 0.66 while cutting compute roughly 3-4x.

Measured effects reported in the paper

Surprising findings

• The KL constraint hurts for the small 1.3B model, contrary to prior findings on larger SD3.5-M models.
• CLIP, known to be weak at counting and attribute binding, still yields gains in these categories under GRPO's relative advantages.

Limitations

• Training is sensitive to learning rate when KL is removed; training collapses at 5e-6 without KL.
• Relies on CLIP as reward model, which is narrow and not fully accurate for RL post-training.
• Only tested on a 1.3B base model; scaling and video generation remain unexplored.

Experimental setup

Datasets: LAION

Benchmarks: GenEval MSCOCO-30K FID ImageReward preference score

Metrics: GenEval overall score FID-30K ImageReward

Baselines: Show-o Janus Meissonic MaskGen-XL Emu3 Infinity SD3.5-L DALLE-3

Builds on: GRPO Show-o MaskGIT DeepSeek-R1

Closest papers by meaning

• Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Image Generation (NeurIPS 2026 · similarity 0.91)
• Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.91)
• MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation (NeurIPS 2026 · similarity 0.91)
• Flow-GRPO: Training Flow Matching Models via Online RL (NeurIPS 2025 · similarity 0.90)
• Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.90)
• Training Diffusion Models with Reinforcement Learning (ICLR 2024 · similarity 0.88)