AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

DisPPO: Quantile-Based Distributional Reinforcement Learning for Large Language Models

Zhijian Zhou, Long Li, Xuan Zhang, Zongkai Liu, Yanting Miao, Yuchen Liu, Deshu Chen, Ke Li, Xing Sun, Ruoxi Jiang, Xiaoyu Tan, Chao Qu and 1 more

ICML 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Distributional value learning

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace scalar critic with quantile distributional criticPPO · Math reasoning, averageavg@k: 43.1 -> 45.0Improved
replace scalar critic with quantile distributional criticPPO · Math reasoning, averagepass@k: 67.3 -> 71.2Improved
replace scalar critic with quantile distributional criticPPO · AIME24pass@256: 73.3 -> 86.7Improved
replace scalar critic with quantile distributional criticPPO · College Mathavg@8: 42.7 -> 43.5Improved
replace scalar critic with quantile distributional criticPPO · Bird (in domain)greedy accuracy (%): 61.4 -> 63.5Improved
replace scalar critic with quantile distributional criticPPO · Spider (out of domain)greedy accuracy (%): 77.0 -> 81.2Improved
replace scalar critic with quantile distributional criticPPO · Multi-turn tool reasoning, averageavg@64: 29.1 -> 30.2Improved
replace scalar critic with quantile distributional criticPPO · RLHF UltraFeedback: AlpacaEval 2win rate (%): 20.77 -> 30.14Improved
replace scalar critic with quantile distributional criticPPO · RLHF UltraFeedback: Arena-Hardscore: 18.3 -> 19.3Improved
replace scalar critic with quantile distributional criticPPO · RLHF UltraFeedback: MT-Benchscore: 7.18 -> 7.41Improved
replace scalar critic with quantile distributional criticPPO · AIME25 value predictionMAE (lower better): 0.3016 -> 0.2749Improved
replace scalar critic with quantile distributional critic, middle MC bin [0.3, 0.7)PPO · AIME25 value prediction by MC binMAE (lower better): 0.1723 -> 0.2346Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy