Research map / Reinforcement learning / RL from feedback
Inpainting-Guided Policy Optimization for Diffusion Large Language Models
Siyan Zhao, Mengchen Liu, Jing Huang, Miao Liu, Chenyu Wang, Bo Liu, Yuandong Tian, Guan Pang, Sean Bell, Aditya Grover, Feiyu Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IGPO RL on LLaDA-Instruct (vs DiffuGRPO) | LLaDA-8B-Instruct · GSM8K | pass@1: 81.9 -> 83.6 | Improved |
| add IGPO RL on LLaDA-Instruct (vs DiffuGRPO) | LLaDA-8B-Instruct · MATH500 | pass@1: 40.2 -> 42.8 | Improved |
| add IGPO RL on LLaDA-Instruct (vs DiffuGRPO) | LLaDA-8B-Instruct · AMC | avg@16: 17.5 -> 18.1 | Improved |
| add Length-aligned SFT on rewritten traces (ablation) | LLaDA-8B-Instruct · MATH500 | pass@1: 39.0 -> 45.2 | Improved |
| add Length-aligned SFT on rewritten traces (ablation) | LLaDA-8B-Instruct · GSM8K | pass@1: 81.5 -> 83.6 | Improved |
| add IGPO after Length-aligned SFT (ablation) | LLaDA-8B-Instruct + Length-aligned SFT · GSM8K | pass@1: 83.6 -> 86.4 | Improved |
| add IGPO after Length-aligned SFT (ablation) | LLaDA-8B-Instruct + Length-aligned SFT · MATH500 | pass@1: 45.2 -> 47.4 | Improved |
| add IGPO after Length-aligned SFT (ablation) | LLaDA-8B-Instruct + Length-aligned SFT · AMC | avg@16: 22.3 -> 24.4 | Improved |
| add inpainting hints to GRPO sampling (ablation) | GRPO on LLaDA · training all-wrong group ratio | fraction of all-wrong groups: reduced by around 60% | Improved |
| use partial hint injection (eta in U[0.2,0.6]) instead of full ground-truth injection (eta=1.0) (ablation) | IGPO · GSM8K, MATH500, AMC | accuracy | Improved |
| add hint injection versus no hints (ablation) | GRPO on LLaDA · GSM8K, MATH500, AMC | accuracy | Improved |
| restrict hint-token gradients to top 20% entropy positions (tau=0.2 vs 1.0) (ablation) | IGPO · GSM8K (temp 0.1, length 256) | accuracy | Improved |
Closest papers by meaning
- d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages (NeurIPS 2026 · similarity 0.90)
- Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.89)
- d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation (ICML 2026 · similarity 0.89)
- Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models (ICLR 2026 · similarity 0.89)
- Diffusion-State Policy Optimization for Masked Diffusion Language Models (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
