Research map / Reinforcement learning / Visual and representation RL
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
Lili Chen, Kimin Lee, Aravind Srinivas, Pieter Abbeel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Freeze the encoder and store latent vectors | Rainbow · Atari games at 45T FLOPs | average return | Improved |
| Freeze the encoder and store latent vectors | Rainbow · BankHeist at 45T FLOPs | average return: 145.8 to 276.6 | Improved |
| Freeze the encoder and store latent vectors | Rainbow · Qbert at 45T FLOPs | average return: 2325.5 to 4123.5 | Improved |
| Freeze the encoder and store latent vectors, increasing replay capacity | Rainbow · Atari games at 500K environment steps, 0.07GB | average return | Mixed |
| Freeze the encoder and store latent vectors, increasing replay capacity | Rainbow · CrazyClimber at 500K environment steps, 0.07GB | average return: 10498.0 to 17620.0 | Improved |
| Freeze the encoder and store latent vectors, increasing replay capacity | Rainbow · Qbert at 500K environment steps, 0.07GB | average return: 2430.5 to 3231.0 | Improved |
| Freeze the encoder after 50000, 100000, or 150000 steps instead of not freezing (ablation) | Rainbow+SEER · Amidar | sample-efficiency: 50000, 100000, 150000 | No clear effect |
| Freeze the encoder earlier (ablation) | Rainbow+SEER · Atari | compute-efficiency: T_f=150000 | Improved |
| Freeze more convolutional layers (ablation) | CURL+SEER transfer from Walker-stand to Walker-walk · Walker-walk transfer | compute-efficiency | Improved |
| Reduce image resolution from 100 × 100 to 50 × 50 (ablation) | CURL · DeepMind Control Suite | compute-efficiency: 100 × 100 to 50 × 50 | Improved |
| Reduce image resolution from 84 × 84 to 42 × 42 (ablation) | Rainbow · Atari games | performance: 84 × 84 to 42 × 42 | Hurt |
| Use a pretrained encoder and freeze it during fine-tuning | CURL · Walker-stand to Walker-walk and Hopper-hop transfer | compute-efficiency: 60K environment interactions | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Rainbow — Rainbow: Combining Improvements in Deep Reinforcement Learning (AAAI 2018) · better on 15 of 16 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Remember and Forget for Experience Replay (ICML 2019 · similarity 0.90)
- Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder (AAAI 2025 · similarity 0.89)
- Pretraining Representations for Data-Efficient Reinforcement Learning (NeurIPS 2021 · similarity 0.89)
- Decoupling Representation Learning from Reinforcement Learning (ICML 2021 · similarity 0.89)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Turning Sand to Gold: Recycling Data to Bridge On-Policy and Off-Policy Learning via Causal Bound (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
