Research map / Image generation / Fast diffusion sampling

Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Image Generation

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Co-GRPO jointly optimizes a masked diffusion image model (Meissonic) and its inference schedule as one MDP with trajectory-level GRPO, alternating model and schedule updates, improving ImageReward and HPSv2 and transferring zero-shot to GenEval and DPG-Bench.

Problem

Masked diffusion models are trained with a single-step BERT-style objective, while inference depends on hand-designed schedules (re-mask ratio, temperatures, guidance scale) that are never optimized, causing a training-inference mismatch.

Approach

Reformulates MDM generation as an MDP whose action includes both the next visual tokens and the inference schedule, factorizes the joint policy into a model policy and a Gaussian schedule policy, and optimizes them with GRPO via alternating updates without backpropagating through the multi-step generation.

Key results

• ImageReward rises from 0.942 (Meissonic) to 1.122 and HPSv2 from 28.83 to 29.37 with Co-GRPO.
• Zero-shot GenEval improves from 0.47 to 0.55 and DPG-Bench from 64.57 to 70.10.
• Co-GRPO at fewer than 16 steps matches the baseline's ImageReward at 48 steps, per the appendix claim of about 4.2x faster inference.

Surprising findings

• Joint optimization of model and schedule is less effective than the alternating strategy (ImageReward 1.031 vs 1.122), suggesting instability from the asymmetry between the 1B model and the small schedule network.

Limitations

• Schedule-policy sensitivity to step count requires interpolation for transfer across step counts.
• Only Meissonic with a Gaussian schedule policy is evaluated; generalization to other MDMs is not shown.
• Reported gains are on reward-model metrics that are also training objectives, so human-preference evaluation is absent.

Experimental setup

Datasets: HPDv2 ImageReward prompts

Benchmarks: ImageReward HPSv2 GenEval DPG-Bench MPS Clip-Score

Metrics: ImageReward score HPSv2 score GenEval score DPG-Bench score MPS CLIP score

Baselines: Naive GRPO Meissonic SDXL Base 1.0 Pixart-alpha DALL-E 3 Show-o SD v2.1 LlamaGen

Builds on: GRPO Meissonic DanceGRPO Mask-GRPO MMaDA

Closest papers by meaning

• Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation (NeurIPS 2025 · similarity 0.91)
• MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation (NeurIPS 2026 · similarity 0.88)
• Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking (NeurIPS 2025 · similarity 0.88)
• Fixed-Point Masked Generative Modeling (NeurIPS 2026 · similarity 0.88)
• Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.88)
• Effective and Efficient Masked Image Generation Models (ICML 2025 · similarity 0.87)