Research map / Reinforcement learning / Visual and representation RL
Robust visual encoders for RL
Make pixel-based RL generalize to visual perturbations via pretrained encoders, invariance, and attention.
86 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Visual and representation RL, keyword cluster “representation · latent · generalization”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
27 papers in 2023–24 and 24 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 4.7% to 3.4%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Deep Reinforcement Learning-Based Image Captioning With Embedding Reward (CVPR 2017 · 314 citations)
- 2019 · Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing (AAAI 2019 · 61 citations)
- 2020 · Reinforcement Learning with Augmented Data (NeurIPS 2020 · 231 citations)
- 2021 · Improving Sample Efficiency in Model-Free Reinforcement Learning from Images (AAAI 2021 · 205 citations)
- 2022 · Mask-based Latent Reconstruction for Reinforcement Learning (NeurIPS 2022 · 21 citations)
- 2023 · Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning (NeurIPS 2023 · 10 citations)
- 2024 · What Effects the Generalization in Visual Reinforcement Learning: Policy Consistency with Truncated Return Prediction (AAAI 2024 · 9 citations)
- 2025 · Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder (AAAI 2025 · 2 citations)
- 2026 · Improving Deepfake Detection with Reinforcement Learning-Based Adaptive Data Augmentation (AAAI 2026 · 1 citation)
Earliest papers here
- Deep Reinforcement Learning-Based Image Captioning With Embedding Reward (CVPR 2017 · 314 citations)
- Learning Visual Servoing with Deep Features and Fitted Q-Iteration (ICLR 2017 · 51 citations)
- Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing (AAAI 2019 · 61 citations)
What has been tried
64 papers here report 727 measured changes (a component added, removed or swapped, compared with a baseline): helped 61%, no clear effect 9%, hurt 23%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Replace the Transformer predictor with a fully connected predictor | DMControl-GB color-hard and video-easy tasks · return | 372 ± 25 vs 313 ± 57; 426 ± 44 vs 355 ± 29 | Hurt | What Effects the Generalization in Visual Reinforcement Learning: Policy Consistency with Truncated Return Prediction (AAAI 2024 · 9 citations) |
| Add contrastive learning | iGibson 1.0 PointGoal, Env Avg, 500k · SR | 5.0 -> 59.7 | Helped | ReCoRe: Regularized Contrastive Representation Learning of World Model (CVPR 2024 · 6 citations) |
| Use more augmented samples (RAD [M=1,K=1] → RAD+ [M=2,K=1] → DrQ [M=2,K=2]) | DeepMind Control Suite, 9 environments, normal background · critic-loss variance | Helped | Revisiting Data Augmentation in Deep Reinforcement Learning (ICLR 2024 · 3 citations) | |
| Add dormant-ratio-guided exploitation | Adroit · final success rate | nearly 20% | Helped | DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · 3 citations) |
| add foreground consistency loss | DMControl video-hard · episode return | -15% | Helped | Focus On What Matters: Separated Models For Visual-Based RL Generalization (NeurIPS 2024 · 3 citations) |
| Use adaptive gaussian-blur augmentation rather than adaptive color-jitter augmentation | ImageNet-100 validation · top-5 classification accuracy | 91.08 -> 91.7 | Helped | Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder (AAAI 2025 · 2 citations) |
| replace the linear encoder layer with an MLP | DMC and ManiSkill2; ID and OOD evaluation · normalized return | No clear effect | The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · 2 citations) | |
| Add prioritized proximal experience regularization | Visual control tasks (2 tasks) · performance | Helped | Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization (NeurIPS 2024 · 2 citations) | |
| Add RAL without ODL | CARLA highway driving · episode reward | Helped | Visual Reinforcement Learning with Residual Action (AAAI 2025 · 1 citation) | |
| add the attention selector | finger-spin · return | Helped | Focus-Then-Decide: Segmentation-Assisted Reinforcement Learning (AAAI 2024 · 1 citation) | |
| Add Reset when data augmentation is absent | DeepMind Control Suite · training performance | Helped | Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages (ICLR 2024 · 1 citation) | |
| Add frame stacking of three consecutive observations | DMC-VB policy learning · reward | Helped | DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 11 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- DeepMind Control Suite · 4 of 11 papers
- Places · 2 of 11 papers
Baselines
- DrQ-v2 · 4 of 11 papers
- SVEA · 4 of 11 papers
- DreamerV3 · 2 of 11 papers
- CURL · 2 of 11 papers
- DrQ · 2 of 11 papers
- SRM · 2 of 11 papers
- PPO · 2 of 11 papers
- RAD · 2 of 11 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 51 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 8 papers
- Chinese Academy of Sciences · 4 papers
- University of Maryland, College Park · 3 papers
- Google DeepMind · 3 papers
- Nanjing University · 2 papers
- Nanjing University of Science and Technology · 2 papers
- Chinese academy of science · 2 papers
- JD Explore Academy · 2 papers
- Shanghai Jiao Tong University · 2 papers
- Microsoft · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Learned state and action representations (148 papers)
- Latent-space world models (83 papers)
- Successor features for transfer (12 papers)
Elsewhere in the topic
- In-context and adaptive RL (202 papers)
- Scalable RL systems and environments (179 papers)
- Probabilistic and autonomous RL (177 papers)
- RL for combinatorial routing (158 papers)
- Dexterous manipulation transfer (141 papers)
- RL with tree search for discovery (121 papers)
- Inferring decision-making from behavior (93 papers)
- Skill learning and diffusion policies (86 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).