Research map / Reinforcement learning / RL from feedback
Policy-Based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards
Orhun Bugra Baran, Melih Kandemir, Ramazan Gokberk Cinbis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune with CLIP, HPSv2 and LOO-FID rewards via GRPO | LlamaGen-L · ImageNet 256 (CFG 1.5) | FID: 4.64 -> 3.83 | Improved |
| fine-tune with proposed rewards without CFG | LlamaGen-XL · ImageNet 256 (no CFG) | FID: 12.66 -> 4.55 | Improved |
| fine-tune with proposed rewards without CFG | LlamaGen-B · ImageNet 256 (no CFG) | FID: 20.89 -> 8.91 | Improved |
| fine-tune with proposed rewards | LlamaGen-B · ImageNet 256 (CFG 1.5) | Recall: 0.61 -> 0.54 | Hurt |
| apply method to VQGAN backbone (no top-k) | VQGAN · ImageNet 256 | FID: 21.90 -> 15.47 | Improved |
| add LOO-FID and adaptive entropy versus AR-GRPO (CLIP+HPSv2) | LlamaGen-L · ImageNet 256 (CFG 1.5) | FID: 6.37 -> 3.83 | Improved |
| add LOO-FID reward to CLIP+HPS rewards (ablation) | LlamaGen-L · ImageNet 256 (CFG 1.5) | FID: 6.21 -> 3.83 | Improved |
| use CLIP and HPS rewards together versus HPS only (ablation) | LlamaGen-L (no FID reward) · ImageNet 256 | IS: 176.73 -> 220.28 | Improved |
| add KL regularization weight 3.0 versus none (ablation) | LlamaGen-L · ImageNet 256 | FID: 10.14 -> 3.83 | Improved |
| add KL regularization to FID-only reward (ablation) | LlamaGen-L FID-only · ImageNet 256 | FID: 3.97 -> 4.07 | No clear effect |
| add entropy regularization 4e-3 versus none (ablation) | LlamaGen-L · ImageNet 256 | FID: 4.96 -> 3.83 | Improved |
| use EMA statistics instead of batch statistics for LOO-FID (ablation) | LlamaGen-L · ImageNet 256 | FID: 4.06 -> 3.83 | Improved |
Closest papers by meaning
- HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation (CVPR 2025 · similarity 0.87)
- David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training (ICML 2025 · similarity 0.87)
- Guiding Visual Autoregressive Models through Spectrum Weakening (NeurIPS 2026 · similarity 0.86)
- Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.86)
- GARDO: Reinforcing Diffusion Models without Reward Hacking (NeurIPS 2026 · similarity 0.86)
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
