Research map / Reinforcement learning / RL from feedback
VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generation
Shikun Sun, Liao Qu, Huichao Zhang, Yiheng Liu, Yangyang Song, Xian Li, Yi Jiang, Xu Wang, Jia Jia, Daniel K. Du, Xinglong Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply RL with VMR prefix/suffix split at m=256 vs vanilla GRPO (m=1024) (ablation) | NextFlow GRPO · CVTG-2K | Word Accuracy: 0.5741 -> 0.6565 | Improved |
| move VMR step earlier from m=512 to m=128 (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6351 -> 0.6677 | Improved |
| choose m=256 instead of m=128 (default, for compute and mask compatibility) (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6677 -> 0.6565 | No clear effect |
| add per-action normalization weighting with exponent 0.6 vs 1.2 (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6602 -> 0.7136 | Improved |
| add Mask Propagation (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6855 -> 0.7071 | Improved |
| add Mask Propagation (ablation) | NextFlow-RL · CVTG-2K | CLIPScore: 0.8188 -> 0.8184 | No clear effect |
| use K=2 rollouts for VMR instead of K=1 (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6575 -> 0.6821 | Improved |
| use K=4 rollouts instead of K=2 (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6821 -> 0.6720 | Hurt |
| fine-grained (3:1) alternation of prefix and suffix updates vs coarse-grained (ablation) | NextFlow-RL · CVTG-2K | Word Accuracy: 0.6778 -> 0.6855 | Improved |
| apply OCR-reward RL with VMR/PANW/MP | NextFlow · CVTG-2K | Word Accuracy: 0.5536 -> 0.7841 | Improved |
| apply HPSv3-reward RL | NextFlow · HPSv3 benchmark (All) | HPS score: 8.43 -> 10.64 | Improved |
Closest papers by meaning
- Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation (NeurIPS 2025 · similarity 0.88)
- Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.87)
- Rendering-Aware Reinforcement Learning for Vector Graphics Generation (NeurIPS 2025 · similarity 0.87)
- Variance-Optimal State Resampling for Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.87)
- Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Image Generation (NeurIPS 2026 · similarity 0.86)
- GVPO: Group Variance Policy Optimization for Large Language Model Post-Training (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
