Research map / Reinforcement learning / RL from feedback
wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models
Xiaohang Tang, Rares Dolga, Sangwoong Yoon, Ilija Bogunovic
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SFT + diffu-GRPO (d1) with wd1 (no SFT) | LLaDA-8B-Instruct · Sudoku (gen len 256) | accuracy: 17.6 -> 76.4 | Improved |
| replace d1 with wd1 | LLaDA-8B-Instruct · Countdown (gen len 256) | accuracy: 25.8 -> 51.2 | Improved |
| replace d1 with wd1 | LLaDA-8B-Instruct · GSM8K (gen len 256) | accuracy: 78.2 -> 80.8 | Improved |
| replace d1 with wd1 | LLaDA-8B-Instruct · MATH500 (gen len 512) | accuracy: 38.0 -> 39.0 | Improved |
| replace d1 with wd1 (no change at length 256) | LLaDA-8B-Instruct · MATH500 (gen len 256) | accuracy: 34.4 -> 34.4 | No clear effect |
| remove SFT stage and use wd1 instead | d1 · training cost | SFT time: 2.01 hrs -> 0 hrs | Improved |
| remove old-policy likelihood approximation (wd1 vs d1) | d1 · RL training time per step | seconds/step: 103.5 -> 81.16 | Improved |
| add negative sample weight w- to wd1 (ablation) | wd1 · Sudoku (gen len 256, ablation checkpoints) | accuracy: 6.69 -> 25.2 | Improved |
| add negative sample weight w- to wd1 (ablation) | wd1 · Countdown (gen len 256, ablation checkpoints) | accuracy: 13.67 -> 51.2 | Improved |
| add negative sample weight w- to wd1 (ablation) | wd1 · GSM8K (gen len 256, ablation checkpoints) | accuracy: 65.66 -> 80.8 | Improved |
| add SFT before wd1 training (ablation) | wd1 · Sudoku (gen len 256, ablation checkpoints) | accuracy: 25.2 -> 26.5 | No clear effect |
| add SFT before wd1 training (ablation) | wd1 · Countdown (gen len 256, ablation checkpoints) | accuracy: 51.2 -> 43.4 | Hurt |
Closest papers by meaning
- d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning (NeurIPS 2025 · similarity 0.93)
- Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages (NeurIPS 2026 · similarity 0.93)
- Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models (ICLR 2026 · similarity 0.91)
- d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation (ICML 2026 · similarity 0.91)
- Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning (ICML 2026 · similarity 0.91)
- MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
