Research map / Reinforcement learning / Visual and representation RL
Improving Sample Efficiency in Model-Free Reinforcement Learning from Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use joint reconstruction and off-policy policy learning with SAC+AE instead of SAC:pixel | SAC · DeepMind Control Suite pixel tasks | evaluation reward | Helped |
| Use SAC+AE instead of forward-model-based methods under moving-background distractors | pixel-based control agents · DMC tasks with moving distractors | evaluation reward | Helped |
| Update the representation more frequently during alternating β-VAE and policy training (ablation) | SAC+VAE:pixel (iter, N) · DMC pixel tasks | evaluation reward | Helped |
| Jointly propagate actor-critic gradients into a β-VAE encoder rather than keeping training separate (ablation) | SAC+VAE:pixel · DMC pixel tasks | evaluation reward | Hurt |
| Replace the stochastic β-VAE with a deterministic regularized autoencoder (ablation) | SAC+AE · DMC pixel tasks | training stability | Helped |
| Block actor gradients from updating the shared convolutional encoder (ablation) | SAC+AE · DMC pixel tasks | evaluation reward | Helped |
| Allow actor gradients to update the shared convolutional encoder (ablation) | SAC+AE · DMC pixel tasks | evaluation reward | Hurt |
| Use low-dimensional state reconstruction instead of pixel reconstruction as the auxiliary target (ablation) | SAC:pixel with state supervision · ball_in_cup_catch | evaluation reward | Hurt |
| Vary autoencoder capacity by changing convolutional layers and channels (ablation) | SAC+AE · DMC pixel tasks | final evaluation reward | No clear effect |
| Initialize the encoder for a new task with weights trained on walker_walk (ablation) | SAC:pixel · walker_stand and walker_run | evaluation reward | Helped |
| Change action repeat from four to one or two (ablation) | SAC+AE · ball_in_cup_catch | evaluation reward | Helped |
| Change action repeat from two to one (ablation) | SAC+AE · walker_walk | evaluation reward | Helped |
Closest papers by meaning
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder (AAAI 2025 · similarity 0.87)
- Towards Principled Representation Learning from Videos for Reinforcement Learning (ICLR 2024 · similarity 0.86)
- VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.86)
- Understanding and Improving Training-free Loss-based Diffusion Guidance (NeurIPS 2024 · similarity 0.86)
- Zero-Shot Reinforcement Learning via Function Encoders (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.