Research map / Reinforcement learning / RL from feedback

Adaptive Robust Estimator for Policy Optimization in Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: GRPO stability and variants

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an adaptive robust estimator to replace the batch mean in GRPO-style advantage normalization so that noisy, skewed, heavy-tailed, or contaminated rewards do not distort policy optimization for LLM reasoning.

Problem

GRPO and its variants normalize rewards by batch-level empirical mean, which is severely distorted by noisy, skewed, heavy-tailed, or contaminated rewards during RL post-training of LLMs.

Approach

Replace the empirical mean in group-relative advantage normalization with an adaptive robust estimator so that the baseline is insensitive to reward outliers and distributional skew.

Experimental setup

Baselines: GRPO

Builds on: GRPO

Closest papers by meaning

• Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
• Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents (ICML 2026 · similarity 0.89)
• Blending Imitation and Reinforcement Learning for Robust Policy Improvement (ICLR 2024 · similarity 0.89)
• Adversarial Diffusion for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• Constrained Reinforcement Learning Under Model Mismatch (ICML 2024 · similarity 0.88)
• TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback (NeurIPS 2026 · similarity 0.88)