AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

Exploration-Guided Reward Shaping for Reinforcement Learning under Sparse Rewards

Rati Devidze, Parameswaran Kamalaruban, Adish Singla

NeurIPS 2022 · Reinforcement learning › Visual and representation RL · approach: Reward shaping for sparse rewards

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add intrinsic bonus (EXPLOB) to intrinsic reward (SELFRS), i.e. full ExploRS (ablation)ExploRS REINFORCE and Q-learning agents · CHAIN0 and CHAIN+expected rewardImproved
add intrinsic reward learning to count-based bonus alone (ExploRS vs EXPLOB) (ablation)count-based exploration bonus · CHAIN, ROOM, LINEKexpected rewardImproved
add exploration bonus to self-supervised intrinsic reward (ExploRS vs SELFRS) (ablation)SELFRS · CHAIN with distractor (CHAIN+)expected rewardImproved
add distractor state with small reward (Rdis=0.01) for SORS', LIRPG', SELFRSSORS', LIRPG', SELFRS · CHAIN+ vs CHAIN0expected rewardHurt
add distractor states (Rdis=0.01) for SORS', LIRPG', SELFRS in neural LINEKSORS', LIRPG', SELFRS · LINEK+expected rewardHurt
use ExploRS instead of all baselines with REINFORCE agenttabular REINFORCE · CHAIN0, CHAIN+expected rewardImproved
use ExploRS instead of baselines with Q-learning agenttabular Q-learning · CHAIN0, CHAIN+expected rewardImproved
use ExploRS instead of baselines on ROOMtabular REINFORCE · ROOM0, ROOM+expected rewardImproved
use ExploRS with neural-network intrinsic reward instead of baselinesneural REINFORCE · LINEK0, LINEK+expected rewardImproved
combine intrinsic reward and bonus in chain theory (SELFRS=1, EXPLOB=1) vs no shaping (ablation)stylized TD-style learner · chain environment (theory)time steps to optimal policyImproved
add only intrinsic bonus (no intrinsic reward) in chain theory (ablation)stylized TD-style learner · chain environment (theory)time steps to optimal policyImproved
add only intrinsic reward (no bonus) in chain theory (ablation)stylized TD-style learner · chain environment (theory)time steps to optimal policyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy