AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models

Chenyu Wang, Paria Rashidinejad, DiJia Su, Song Jiang, Sid Wang, Siyan Zhao, Cai Zhou, Shannon Zejiang Shen, Feiyu Chen, Tommi Jaakkola, Yuandong Tian, Bo Liu

ICLR 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning · 45 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the negative advantage loss (ablation)SPG · GSM8K/MATH500/Countdown/Sudoku (avg lengths)accuracy: 77.4 -> 83.1 (GSM8K); 32.7 -> 38.4 (MATH500); 45.5 -> 69.9 (Countdown); 68.8 -> 90.0 (Sudoku) with MixtureImproved
use EUBO for negative advantage traces instead of ELBO (ablation)SPG · GSM8K/Countdown/Sudoku (avg lengths)accuracy: 80.9 -> 81.6; 67.1 -> 69.3; 82.4 -> 86.1Improved
use EUBO for negative advantage traces instead of ELBO (ablation)SPG · MATH500 (avg lengths)accuracy: 37.4 -> 36.7Hurt
use Mixture of ELBO and EUBO instead of ELBO for negative traces (ablation)SPG · four reasoning benchmarks (avg lengths)accuracy: +2.2 GSM8K, +1.0 MATH500, +2.8 Countdown, +7.6 SudokuImproved
use block-wise masking instead of random masking (ablation)SPG w/ EUBO · Countdown (avg lengths)accuracy: 45.4 -> 69.3Improved
use block-wise masking instead of random masking (ablation)SPG w/ EUBO · MATH500 (avg lengths)accuracy: 36.7 -> 36.7No clear effect
use block-wise masking instead of random masking (ablation)SPG w/ Mixture · MATH500 and Countdown (avg lengths)accuracy: 36.9 -> 38.4; 62.8 -> 69.9Improved
mix lower and upper bounds with omega in (0,1) (ablation)SPG · benchmarksaccuracyImproved
use intermediate beta between 0 and 1 (ablation)SPG · benchmarksaccuracyHurt
replace UniGRPO with SPG w/ MixturedLLM RL (LLaDA-8B-Instruct) · GSM8K/MATH500/Countdown/Sudoku (len 256)accuracy: 82.5 -> 86.1; 37.4 -> 40.0; 43.0 -> 70.7; 67.0 -> 94.0Improved
replace UniGRPO with SPG w/ MixturedLLM RL (LLaDA-8B-Instruct) · HumanEval (len 256)accuracy: 39.6 -> 39.0Hurt
replace UniGRPO with SPG w/ MixturedLLM RL (LLaDA-8B-Instruct) · MBPP (len 256)accuracy: 45.5 -> 50.6Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy