Research map / Reinforcement learning / Visual and representation RL
Retrieval-Augmented Reinforcement Learning
Anirudh Goyal, Abram Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adrià Puigdomènech Badia, Arthur Guez, Mehdi Mirza, Peter C Humphreys, Ksenia Konyushova, Michal Valko, Simon Osindero and 3 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add retrieval augmentation to R2D2 | R2D2 · Atari | mean human normalized score: 11.32 ± 1.2% | Improved |
| Add task-specific retrieval to RDQN | RDQN · BabyAI (40 levels, 50K) | success rate: 32% ± 4% -> 48% ± 4% | Improved |
| Use a multi-task retrieval buffer instead of a task-specific buffer | RA-RDQN · BabyAI (40 levels, 50K) | success rate: 48% ± 4% -> 55% ± 5% | Improved |
| Add task-specific retrieval to RDQN | RDQN · BabyAI (40 levels, 200K) | success rate: 45% ± 6% -> 64% ± 5% | Improved |
| Use a multi-task retrieval buffer instead of a task-specific buffer | RA-RDQN · BabyAI (40 levels, 200K) | success rate: 64% ± 5% -> 74% ± 3% | Improved |
| Add task-specific retrieval to behavior cloning | BC · CausalWorld (5 tasks, 50K) | success rate: 61% ± 10% -> 71% ± 7% | Improved |
| Use a multi-task retrieval buffer instead of a task-specific buffer | RA-BC · CausalWorld (5 tasks, 50K) | success rate: 71% ± 7% -> 82% ± 5% | Improved |
| Use a separate retrieval state and process rather than querying directly from the agent state (ablation) | RA-R2D2 · Atari | performance relative to R2D2 | Improved |
| Retrieve information from the experience dataset (ablation) | RA-R2D2 · Atari | performance relative to R2D2 | Improved |
| Summarize retrieval trajectories with a context length of 80 rather than 5 timesteps (ablation) | RA-R2D2 · Atari | performance | Improved |
| Add BERT-style masking auxiliary loss alongside action, reward, and value prediction losses (ablation) | RA-R2D2 · Atari | performance | Improved |
| Use only BERT-style masking loss instead of action, reward, and value prediction losses (ablation) | RA-R2D2 · Atari | performance | Hurt |
Closest papers by meaning
- REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environments (ICLR 2025 · similarity 0.87)
- Understanding when Dynamics-Invariant Data Augmentations Benefit Model-free Reinforcement Learning Updates (ICLR 2024 · similarity 0.87)
- Prioritized Generative Replay (ICLR 2025 · similarity 0.87)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.87)
- Turning Sand to Gold: Recycling Data to Bridge On-Policy and Off-Policy Learning via Causal Bound (NeurIPS 2025 · similarity 0.87)
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
