AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Group Distributionally Robust Optimization-Driven RL for LLM Reasoning

Kishan Panaganti, Zhenwen Liang, Wenhao Yu, Haitao Mi, Dong Yu

ICML 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Casts GRPO post-training as two independent group distributionally robust optimization (GDRO) games over dynamically defined difficulty groups, targeting heavy-tailed hard reasoning problems.

Problem

GRPO reasoning post-training uses static uniform prompt sampling and a fixed number of rollouts per prompt, which wastes compute on solved patterns and under-trains the long tail of hard problems in heterogeneous, heavy-tailed reasoning data.

Approach

Replaces static uniformity with two independent GDRO games over dynamic difficulty groups defined by an online training signal, adapting both prompt sampling and rollout allocation toward hard groups.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Prompt-GDRO adversarial prompt reweightingGRPO · Qwen3-4B-Base, DAPO-14.1kpass@8: 56.31 -> 63.70 (+13.13%)Improved
add Prompt-GDROGRPO · Qwen3-1.7B-Basepass@8: 50.74 -> 55.68 (+9.74%)Improved
add Prompt-GDROGRPO · Qwen3-8B-Basepass@8: 62.04 -> 67.60 (+8.96%)Improved
add Rollout-GDRO adaptive rollout allocationGRPO · Qwen3-1.7B-Basepass@8: 50.74 -> 56.14 (+10.64%)Improved
add Rollout-GDROGRPO · Qwen3-8B-Basepass@8: 62.04 -> 67.75 (+9.20%)Improved
rerun Prompt-GDRO with a second seed (ablation)Prompt-GDRO · Qwen3-4B-Basepass@8: 56.36 -> 63.47Improved
vary bins, history, adversary LR, weight cap (ablation)Prompt-GDRO · Qwen3-4B-Basepass@8: within -1.31 to +0.41 pp of 63.70No clear effect
use online Rollout-GDRO instead of fixed hard-focused static rollout schedule (ablation)Rollout-GDRO · Qwen3-4B-Basepass@8: 56.70 -> 62.27Improved
focus fixed rollouts on hard instead of easy prompts (ablation)StaticRollout · Qwen3-4B-Basepass@8: 54.34 -> 56.70Improved
vary bins, arms, dual LR (ablation)Rollout-GDRO · Qwen3-4B-Basepass@8: 0.81 pp spreadNo clear effect
add Rollout-GDRO (second seed) (ablation)GRPO · Qwen3-4B-Base MINERVAmean@8: 30.48 -> 27.28Hurt
add Prompt-GDROGRPO · Qwen3-1.7B-Base MINERVAmean@8: 14.56 -> 14.61No clear effect

Experimental setup

Baselines: GRPO

Builds on: GRPO

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy