AI Paper Atlas

Research map / Reinforcement learning / Offline RL

False Correlation Reduction for Offline Reinforcement Learning

Zhi‐Hong Deng, Zuyue Fu, Lingxiao Wang, Zhuoran Yang, Chenjia Bai, Tianyi Zhou, Zhaoran Wang, Jing Bo Jiang

TPAMI 2023 · Reinforcement learning › Offline RL · approach: Pessimism and uncertainty in offline RL · 15 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add annealing behavior cloning regularizer (ablation)SCORE · hopper-mediumuncertainty stability / returnImproved
add BC regularizer on diverse data (ablation)SCORE · hopper-medium-replayuncertainty stabilityImproved
use BC regularizer instead of VAE-fitted behavior policy (Fix or MMD) (ablation)SCORE · D4RL medium-replay and mediumaverage returnImproved
add uncertainty (pessimism) penalty (ablation)SCORE · hopper-medium-replay and walker2d-medium-replayaverage returnImproved
use bootstrapped ensemble instead of MC dropout (ablation)SCORE · D4RL medium-replayaverage returnImproved
use ensemble penalty instead of min-Q pessimism (ablation)SCORE · D4RL medium-replayaverage returnImproved
penalty on u(s,a), u(s',a') or both (ablation)SCORE · D4RL hopper/walker2daverage returnNo clear effect
increase penalty coefficient beta to 2.0 (ablation)SCORE · hopper/walker2d medium-replayaverage return: beta 0.1/0.2/0.5 similar; 2.0 poorHurt
beta 0.1, 0.2 or 0.5 (ablation)SCORE · D4RLaverage returnNo clear effect
constant BC weight (gamma_bc 1.0) instead of decay (ablation)SCORE · medium-replayaverage return: gamma_bc 0.96/0.98 vs 1.0Hurt
gamma_bc 0.96 on medium dataset (ablation)SCORE · mediumaverage return: 0.98 best; 0.96 collapses lateHurt
increase ensemble size from M=3 to M=5 (ablation)SCORE · D4RLperformance stability: M in {3,5,7,9}Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy