Research map / Reinforcement learning / Visual and representation RL
Contrastive Retrospection: honing in on critical steps for rapid learning and generalization in RL
Chen Sun, Wannan Yang, Thomas Jiralerspong, Dane Malenfant, Benjamin Alsbury-Nealy, Yoshua Bengio, Blake Aaron Richards
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ConSpec intrinsic reward to PPO | PPO · 3D OrangeTree | episode reward | Improved |
| add Synthetic Returns to PPO | PPO · 3D OrangeTree | episode reward | No clear effect |
| add CURL to PPO | PPO · 3D OrangeTree | episode reward | No clear effect |
| test with never-seen black flat objects (zero-shot) | PPO+ConSpec · OrangeTree OoD colour | episode reward | Improved |
| test in a new room with different wall and floor colours after at most 20 PPO gradient steps | PPO+ConSpec · OrangeTree OoD room | episode reward | Improved |
| increase number of keys in multi-key-to-door task | SynthRs, TVT, RUDDER, RND · multi-key-to-door | success rate | Hurt |
| increase number of keys in multi-key-to-door task | PPO+ConSpec · multi-key-to-door | training time excluding exploration | No clear effect |
| use 3 prototypes instead of 4 in the 4-key task (ablation) | PPO+ConSpec · 4-key-to-door | success rate: 4 -> 3 prototypes | No clear effect |
| remove prototype freezing (ablation) | PPO+ConSpec · multi-key-to-door | success rate | No clear effect |
| use policy-invariant intrinsic reward instead of simple max-score reward | PPO+ConSpec · multi-key-to-door | success rate | No clear effect |
| change intrinsic reward scale lambda from 0.2 to 0.5 (ablation) | PPO+ConSpec · OrangeTree, 4-key, Montezuma's Revenge | episode reward: 0.2 -> 0.5 | No clear effect |
| add ConSpec to PPO on delayed Atari | PPO · delayed Pong, Bowling, AirRaid | reward | Improved |
Closest papers by meaning
- ContraDiff: Planning Towards High Return States via Contrastive Learning (ICLR 2025 · similarity 0.87)
- Learning Process Rewards via Visitation Matching for Efficient RL (NeurIPS 2026 · similarity 0.87)
- A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals (ICLR 2025 · similarity 0.87)
- Discovering Multiple Solutions from a Single Task in Offline Reinforcement Learning (ICML 2024 · similarity 0.85)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- Stitching Sub-trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
