AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

Bridging RL Theory and Practice with the Effective Horizon

Cassidy Laidlaw, Stuart Russell, Anca Dragan

NeurIPS 2023 · Reinforcement learning › MDPs and planning · approach: Policy optimization theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
predict PPO sample complexity with effective horizon bound instead of EPW boundsample complexity bounds · BRIDGE, PPOSpearman correlation: 0.69 -> 0.81Improved
predict PPO sample complexity with effective horizon bound instead of covering lengthsample complexity bounds · BRIDGE, PPOSpearman correlation: 0.35 -> 0.81Improved
predict DQN sample complexity with effective horizon bound instead of EPWsample complexity bounds · BRIDGE, DQNSpearman correlation: 0.58 -> 0.74Improved
predict PPO convergence with effective horizon instead of worst-case boundsample complexity bounds · BRIDGE, PPOAUROC: 0.57 -> 0.92Improved
predict DQN convergence with effective horizon instead of EPWsample complexity bounds · BRIDGE, DQNaccuracy: 0.85 -> 0.86Improved
use effective horizon instead of UCB bound for absolute tightnesssample complexity bounds · BRIDGE, PPOmedian ratio: 20 -> 31Hurt
use effective horizon instead of EPW for absolute tightnesssample complexity bounds · BRIDGE, PPOmedian ratio: $1.1 \times 10^{5}$ -> 31Improved
use effective horizon to predict effect of reward shaping instead of EPWeffective horizon bound · 77 shaped MiniGrid envs, PPOcorrelation of predicted vs empirical change: 0.20 -> 0.48Improved
use effective horizon to predict effect of reward shaping instead of EPWeffective horizon bound · 77 shaped MiniGrid envs, DQNcorrelation of predicted vs empirical change: 0.70 -> 0.35Hurt
use effective horizon instead of covering length to predict effect of pretrained-policy initeffective horizon bound · 82 MDPs, PPOcorrelation of predicted vs empirical change: -0.36 -> 0.57Improved
predict PPO sample complexity with effective horizon bound instead of GORP empiricalempirical proxy · BRIDGE, PPOSpearman correlation: 0.79 -> 0.81Improved
predict DQN sample complexity from PPO empirical results instead of effective horizonempirical proxy · BRIDGE, DQNSpearman correlation: 0.74 -> 0.77Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy