Research map / Reinforcement learning / Visual and representation RL
Which Mutual-Information Representation Learning Objectives are Sufficient for Control?
Kate Rakelly, Abhishek Gupta, Carlos Florensa, Sergey Levine
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| maximize forward information, including the action in the prediction objective | state representation learning for RL · MDPs with state-based rewards | Q*-sufficiency | Improved |
| omit the action from the transition-prediction objective | state representation learning with J_state · Didactic state-only counterexample MDP | return: 0.5 versus 1.0 | Hurt |
| maximize inverse information about actions from state transitions | state representation learning with J_inv · Didactic inverse-information counterexample MDP | return: 0.5 versus 1.0 | Hurt |
| add immediate-reward information I(R_t; Z_t) to inverse information | J_inv representation learning · Didactic inverse-information counterexample MDP | Q*-sufficiency | No clear effect |
| learn the representation by maximizing inverse information | catcher · catcher | expected return | Hurt |
| learn the representation by maximizing state-only transition information | catcher-grip · catcher-grip | expected return | Hurt |
| use a sufficient forward-information representation instead of an insufficient objective | visual representation learning for RL · catcher and catcher-grip with background distractors | expected return | Improved |
| collect representation-learning data with the optimal policy instead of a uniform random policy (ablation) | J_inv representation learning · catcher | expected return | No clear effect |
| add temporally correlated, moving visual distractors (ablation) | representation learning in catcher · catcher with temporally correlated visual distractors | expected return | Mixed |
| regress the learned Q-function directly onto ground-truth Q* instead of using temporal-difference bootstrapping (ablation) | RL using MI-learned representations · catcher and catcher-grip | policy performance | No clear effect |
Closest papers by meaning
- On the Role of Information Structure in Reinforcement Learning for Partially-Observable Sequential Teams and Games (NeurIPS 2024 · similarity 0.86)
- How to Explore with Belief: State Entropy Maximization in POMDPs (ICML 2024 · similarity 0.86)
- Provable Representation with Efficient Planning for Partially Observable Reinforcement Learning (ICML 2024 · similarity 0.86)
- Provable Partially Observable Reinforcement Learning with Privileged Information (NeurIPS 2024 · similarity 0.86)
- Skill-aware Mutual Information Optimisation for Zero-shot Generalisation in Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Online Laplacian-Based Representation Learning in Reinforcement Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
