Research map / Reinforcement learning / Offline RL

Towards Hyperparameter-free Policy Selection for Offline Reinforcement Learning

NeurIPS 2021 · Reinforcement learning › Offline RL · approach: Offline RL sample complexity theory · 3 citations (OpenAlex)

Read the paper Check your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changed	Where	Measured	Effect
Use BVFT instead of 1-sample BR for policy ranking	Offline policy selection · Atari and Gym control	top-k precision and top-k regret	Helped
Automatically select the discretization resolution by minimizing BVFT loss	BVFT · Atari and Gym control	top-k precision and top-k regret	No clear effect
Use BVFT instead of AvgQ for policy ranking	Offline policy selection · Acrobot and Seaquest	top-k precision and top-k regret	Helped
Use AvgQ instead of BVFT for policy ranking	Offline policy selection · Pendulum, LunarLander, and Pong	top-k precision and top-k regret	No clear effect
Use BVFT rather than FQE with limited selection data	Offline policy selection · Atari	top-2 precision and regret	Helped
Reduce FQE selection data from the large-sample regime	FQE · Atari	top-2 precision and regret	Hurt
Apply BVFT-based selection to actor-critic critics without OPE refitting	BVFT and its continuous-action variants · MuJoCo	top-k precision and top-k regret	No clear effect
Refit policy value functions with multiple FQE instances and select among them using BVFT-PE-Q	BVFT-PE-Q policy selection · MuJoCo	top-k precision and top-k regret	Helped
Refit policy value functions with multiple FQE instances and select within each policy using BVFT-PE	BVFT-PE policy selection · MuJoCo	top-k precision and top-k regret	Helped
Use BVFT-PE-Q rather than the per-policy BVFT-PE selection strategy	BVFT-based FQE selection · MuJoCo	top-k precision and top-k regret	Helped
Tune FQE hyperparameters per policy using BVFT-PE instead of using the best fixed hyperparameters	FQE · HalfCheetah, Hopper, and Walker2D	OPE accuracy	Helped
Replace BCQ candidate training with CQL or online DQN candidate training (ablation)	BVFT policy selection · Atari	top-k precision and top-k regret	No clear effect

Closest papers by meaning

Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.93)
A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.89)
Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic (ICML 2024 · similarity 0.89)
Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft