Research map / Reinforcement learning / RL from feedback

Tapered Off-Policy REINFORCE - Stable and efficient reinforcement learning for large language models

NeurIPS 2025 · Reinforcement learning › RL from feedback · approach: GRPO stability and variants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Tapered Off-Policy REINFORCE (TOPR), an asymmetric tapered importance-sampling variant of REINFORCE for stable, efficient LLM fine-tuning that works offline and without KL regularization.

Problem

Fine-tuning large language models with reinforcement learning in a stable and sample-efficient way, including handling positive and negative examples and off-policy/offline data.

Approach

Uses an asymmetric, tapered variant of importance sampling within a REINFORCE-style objective so that off-policy positive and negative examples can be combined in one framework, speeding learning while keeping dynamics stable without KL regularization.

Experimental setup

Builds on: REINFORCE

Closest papers by meaning

• Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards (NeurIPS 2025 · similarity 0.92)
• VESPO: Variational Sequence-level Soft Policy Optimization for Off-Policy LLM Training (NeurIPS 2026 · similarity 0.90)
• Reinforcement Mid-Training (ICLR 2026 · similarity 0.89)
• ROLLVERIFY: BRIDGING EFFICIENCY AND ACCURACY IN LONG-TAIL ROLLOUT REINFORCEMENT LEARNING (NeurIPS 2026 · similarity 0.89)
• On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.89)
• Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models (ICLR 2025 · similarity 0.89)