Research map / Reinforcement learning / RL from feedback

MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: RL fine-tuning of generative models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MaskSense, an RL framework for masked generative models that adds semantic-anchored routing sampling and entropy-transition-anchored gradient selection to GRPO, improving Meissonic on GenEval and HPS.

Problem

GRPO adapted to masked generative models (MGMs) suffers because confidence-based MaskGIT decoding collapses rollout diversity early (background resolved first, subject tokens suppressed), and uniform GRPO updates across all decoding steps ignore uneven step importance.

Approach

Identifies a "visual exploration trap" in confidence-based decoding and counters it with a semantic-prior (cross-attention-derived) sampler that anchors subject tokens early, stochastic routing of each rollout group between this sampler and MaskGIT, and optimizing GRPO only at the top-k steps with largest global entropy transitions, using random remasking for those anchor states.

Key results

• GenEval overall improves from 0.54 (Meissonic) to 0.86 with MaskSense, versus 0.72 for GRPO and 0.76 for MaskFocus.
• HPS rises from 28.89 to 36.39, the best among compared methods (MaskFocus 35.52, MaskGRPO 35.48).
• Applied to Show-o, MaskSense reaches 30.09 HPS at 2K steps vs 29.81 for Mask-GRPO, with about 3.1% extra GPU hours.

Surprising findings

• Sampler identity does not determine advantage sign: branch reward ranges overlap for 90.7% of prompts, and 26.9% of SA-Sampler rollouts fall below the pooled mean.
• Restricting anchors to early steps (first 10% or 20%) gives faster initial gains but ultimately leads to training collapse.

Limitations

• Evaluated only on the MaskGIT-style paradigm and on Meissonic and Show-o; generalization to other MGM architectures is untested.
• Only image generation is studied; extension to multimodal discrete diffusion is left to future work.
• Results rely on rule-based GenEval rewards and HPS, with the VQA judge being a single model (Qwen-VL-Plus).

Experimental setup

Datasets: GenEval prompts (Flow-GRPO) HPS v2 dataset

Benchmarks: GenEval HPS v2 DrawBench Show-o HPS transfer

Metrics: GenEval accuracy HPS DeQA PickScore ImageReward

Baselines: Meissonic GRPO MaskGRPO MaskFocus Mask-GRPO Show-o SD3.5-M FLUX.1-dev

Builds on: GRPO MaskGIT Mask-GRPO MaskFocus

Closest papers by meaning

• Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation (NeurIPS 2025 · similarity 0.91)
• Visual Generation Without Guidance (ICML 2025 · similarity 0.88)
• Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Image Generation (NeurIPS 2026 · similarity 0.88)
• Plug-and-Play Context Feature Reuse for Efficient Masked Generation (NeurIPS 2025 · similarity 0.88)
• Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.87)
• GARDO: Reinforcing Diffusion Models without Reward Hacking (NeurIPS 2026 · similarity 0.87)