AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Ratio-Variance Regularized Policy Optimization

Yu Luo, Shuo Han, Yihan Hu, Lei Lv, Huaping Liu, Fuchun Sun, Jianye HAO, Dong Li

ICML 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace hard clipping (GRPO) with ratio-variance regularization (R2VPO-OFF)GRPO · Math reasoning macro-average over 7 LLMsaccuracy: 42.50 -> 46.33Improved
use on-policy R2VPO instead of GRPOGRPO · Qwen3-1.7B (fast)accuracy: 30.70 -> 37.37Improved
use on-policy R2VPO instead of GRPOGRPO · DeepSeek-Distill-Qwen-1.5Baccuracy: 29.23 -> 29.45No clear effect
add replay buffer off-policy reuse to R2VPOR2VPO-ON · Math reasoning, 7 LLMsaccuracy: 44.90 -> 46.33Improved
add off-policy replay to R2VPO on openPangu-7BR2VPO-ON · openPangu-7Baccuracy: 57.55 -> 56.32Hurt
increase replay buffer capacity (2, 4, 8) with GRPO (ablation)GRPO · Qwen3-1.7B (fast)test scoreHurt
increase replay buffer capacity (2, 4, 8) with R2VPO-OFF (ablation)R2VPO-OFF · Qwen3-1.7B (fast)test scoreNo clear effect
use adaptive dual update for lambda instead of fixed lambda=0.04 (ablation)R2VPO-OFF · Qwen3-8B (fast)test scoreImproved
use fixed lambda=0.04 variance penalty instead of GRPO-CH clipping (ablation)GRPO-CH · Qwen3-8B (fast)test scoreImproved
use R2VPO instead of PPO clippingPPO · DeepMind Control Suite, 10 tasksaverage score: 54.34 -> 65.11Improved
use R2VPO instead of PPO on sparse-reward Ball in CupPPO · DMC BallInCuprewardImproved
use R2VPO instead of PPO on Cheetah RunPPO · DMC CheetahRunrewardImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy