ICLR 2025 accepted papers on Reinforcement learning

Visual and representation RL · 76 papers

• DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search (3 citations)
• AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learning (1 citation)
• BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shaping (1 citation)
• Causal Information Prioritization for Efficient Reinforcement Learning (1 citation)
• Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling (1 citation)
• ET-SEED: EFFICIENT TRAJECTORY-LEVEL SE(3) EQUIVARIANT DIFFUSION POLICY (1 citation)
• Generating Freeform Endoskeletal Robots (1 citation)
• Hierarchical World Models as Visual Whole-Body Humanoid Controllers (1 citation)
• Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning (1 citation)
• Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks (1 citation)
• ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks (1 citation)
• MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization (1 citation)

and 64 more in the interactive map.

Policy gradient and actor-critic · 45 papers

• Conflict-Averse Gradient Aggregation for Constrained Multi-Objective Reinforcement Learning (1 citation)
• Diffusion Policy Policy Optimization (1 citation)
• Monte Carlo Planning with Large Language Model for Text-Based Game Agents (1 citation)
• Simplifying Deep Temporal Difference Learning (1 citation)
• Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation (1 citation)
• $q$-exponential family for policy optimization
• Action abstractions for amortized sampling
• ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning
• Adaptive $Q$-Network: On-the-fly Target Selection for Deep Reinforcement Learning
• Adaptive teachers for amortized samplers
• Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks
• Bisimulation Metric for Model Predictive Control

and 33 more in the interactive map.

Offline RL · 41 papers

• Data Center Cooling System Optimization Using Offline Reinforcement Learning (7 citations)
• Adding Conditional Control to Diffusion Models with Reinforcement Learning (1 citation)
• Any-step Dynamics Model Improves Future Predictions for Online and Offline Reinforcement Learning (1 citation)
• Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning (1 citation)
• Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data (1 citation)
• Learning on One Mode: Addressing Multi-modality in Offline Reinforcement Learning (1 citation)
• Model-based Offline Reinforcement Learning with Lower Expectile Q-Learning (1 citation)
• On Rollouts in Model-Based Reinforcement Learning (1 citation)
• Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (1 citation)
• Towards General-Purpose Model-Free Reinforcement Learning (1 citation)
• AdaWM: Adaptive World Model based Planning for Autonomous Driving
• Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning

and 29 more in the interactive map.

RL from feedback · 28 papers

• Training Language Models to Self-Correct via Reinforcement Learning (7 citations)
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (3 citations)
• Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking (1 citation)
• Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF (1 citation)
• Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design (1 citation)
• Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning (1 citation)
• Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization (1 citation)
• Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference (1 citation)
• A General Framework for Off-Policy Learning with Partially-Observed Reward
• A Theoretical Framework for Partially-Observed Reward States in RLHF
• Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
• Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization

and 16 more in the interactive map.

Imitation learning · 15 papers

• Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling (1 citation)
• Instant Policy: In-Context Imitation Learning via Graph Diffusion (1 citation)
• Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
• Contractive Dynamical Imitation Policies for Efficient Out-of-Sample Recovery
• Diverse Policies Recovering via Pointwise Mutual Information Weighted Imitation Learning
• Efficient Imitation under Misspecification
• Noise-conditioned Energy-based Annealed Rewards (NEAR): A Generative Framework for Imitation Learning from Observation
• Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
• Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
• AutoCGP: Closed-Loop Concept-Guided Policies from Unlabeled Demonstrations
• Generalized Behavior Learning from Diverse Demonstrations
• PN-GAIL: Leveraging Non-optimal Information from Imperfect Demonstrations

and 3 more in the interactive map.

MDPs and planning · 10 papers

• Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis (7 citations)
• Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes (1 citation)
• How to Find the Exact Pareto Front for Multi-Objective MDPs? (1 citation)
• Computationally Efficient RL under Linear Bellman Completeness for Deterministic Dynamics
• Learning a Fast Mixing Exogenous Block MDP using a Single Trajectory
• Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Form
• Sequential Stochastic Combinatorial Optimization Using Hierarchal Reinforcement Learning
• Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs
• Efficient Causal Decision Making with One-sided Feedback
• Global Convergence of Policy Gradient in Average Reward MDPs