AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

Which Mutual-Information Representation Learning Objectives are Sufficient for Control?

Kate Rakelly, Abhishek Gupta, Carlos Florensa, Sergey Levine

NeurIPS 2021 · Reinforcement learning › Visual and representation RL · approach: Latent-space world models · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
maximize forward information, including the action in the prediction objectivestate representation learning for RL · MDPs with state-based rewardsQ*-sufficiencyImproved
omit the action from the transition-prediction objectivestate representation learning with J_state · Didactic state-only counterexample MDPreturn: 0.5 versus 1.0Hurt
maximize inverse information about actions from state transitionsstate representation learning with J_inv · Didactic inverse-information counterexample MDPreturn: 0.5 versus 1.0Hurt
add immediate-reward information I(R_t; Z_t) to inverse informationJ_inv representation learning · Didactic inverse-information counterexample MDPQ*-sufficiencyNo clear effect
learn the representation by maximizing inverse informationcatcher · catcherexpected returnHurt
learn the representation by maximizing state-only transition informationcatcher-grip · catcher-gripexpected returnHurt
use a sufficient forward-information representation instead of an insufficient objectivevisual representation learning for RL · catcher and catcher-grip with background distractorsexpected returnImproved
collect representation-learning data with the optimal policy instead of a uniform random policy (ablation)J_inv representation learning · catcherexpected returnNo clear effect
add temporally correlated, moving visual distractors (ablation)representation learning in catcher · catcher with temporally correlated visual distractorsexpected returnMixed
regress the learned Q-function directly onto ground-truth Q* instead of using temporal-difference bootstrapping (ablation)RL using MI-learned representations · catcher and catcher-grippolicy performanceNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy