Research map / Reinforcement learning / Policy gradient and actor-critic

Distribution-Adaptive Policy Optimization

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper proposes Distribution-Adaptive Policy Optimization, an RLVR method for LLM reasoning that addresses the policy discrepancy introduced by importance-sampling-based off-policy training.

Problem

On-policy RLVR for LLM reasoning is stable but sample-inefficient, while off-policy RL with importance sampling introduces a policy discrepancy between the behavior and target policies.

Approach

Adapts the optimization to the distribution of policy discrepancy so that off-policy training with importance sampling remains stable; the abstract is truncated, so the specific mechanism is not stated.

Closest papers by meaning

• Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.92)
• Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping (ICLR 2026 · similarity 0.91)
• Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards (NeurIPS 2025 · similarity 0.91)
• Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.89)
• VESPO: Variational Sequence-level Soft Policy Optimization for Off-Policy LLM Training (NeurIPS 2026 · similarity 0.89)
• Stabilizing Policy Optimization via Logits Convexity (NeurIPS 2026 · similarity 0.89)