Research map / Reinforcement learning / Visual and representation RL
Graph search and compositional goals
Plan over goal graphs or compositional goal abstractions to improve goal-conditioned RL.
55 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Visual and representation RL, keyword cluster “goal · conditioned · hierarchical”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
15 papers in 2023–24 and 19 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 2.6% to 2.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · Planning with Goal-Conditioned Policies (NeurIPS 2019 · 44 citations)
- 2020 · Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement (NeurIPS 2020 · 33 citations)
- 2021 · Goal-Conditioned Reinforcement Learning with Imagined Subgoals (ICML 2021 · 24 citations)
- 2022 · Contrastive Learning as Goal-Conditioned Reinforcement Learning (NeurIPS 2022 · 24 citations)
- 2023 · Metric Residual Network for Sample Efficient Goal-Conditioned Reinforcement Learning (AAAI 2023 · 5 citations)
- 2024 · Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data (ICLR 2024 · 2 citations)
- 2025 · Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations (NeurIPS 2025 · 2 citations)
- 2026 · QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RL (ICML 2026 · 0 citations)
Earliest papers here
- Planning with Goal-Conditioned Policies (NeurIPS 2019 · 44 citations)
- Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement (NeurIPS 2020 · 33 citations)
- Sub-Goal Trees a Framework for Goal-Based Reinforcement Learning (ICML 2020 · 7 citations)
What has been tried
43 papers here report 429 measured changes (a component added, removed or swapped, compared with a baseline): helped 56%, no clear effect 11%, hurt 28%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| remove T-invariance loss | pointmaze_teleport_stitch · success rate (%) | 29.3(±2.2) -> 18.5(±2.1) | Hurt | Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations (NeurIPS 2025 · 2 citations) |
| use a 3-layer CNN instead of ResNet 18 | Simulated manipulation, drawer · success rate | 28% -> 81% | Helped | Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data (ICLR 2024 · 2 citations) |
| add pre-training from demonstrations | VVC NMR (D0) · success rate (%) | 0.04±0.03 -> 45.87±3.09 | Helped | Goal-Conditioned On-Policy Reinforcement Learning (NeurIPS 2024 · 2 citations) |
| Remove variance loss from PLDM | Two-Rooms, optimal setting · success rate | 98.0 ± 1.5 -> 13.4 ± 9.2 | Hurt | Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models (NeurIPS 2025 · 1 citation) |
| Add offline Q-learning (step 2) instead of using goal-conditioned behavioral cloning | BabyAI Synth, train goals · success rate | 7 to 16 | Helped | The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data (ICML 2025 · 0 citations) |
| Remove hindsight experience replay (HER) | LTL tasks · performance | Hurt | Automaton Constrained Q-Learning (NeurIPS 2025 · 0 citations) | |
| Train SAW target subpolicy on HIQL subgoal representations instead of observations | giant maze environments · success rate | Hurt | Flattening Hierarchies with Policy Bootstrapping (NeurIPS 2025 · 0 citations) | |
| Add MEGA goal selection to DRAG | Success coverage · success coverage | No clear effect | Imagine Beyond ! Distributionally Robust Autoencoding for State Space Coverage in Online Reinforcement Learning (NeurIPS 2025 · 0 citations) | |
| use n-step TD with the original γ instead of 1-step TD | AntMaze-large-explore (ALE) · success rate | 4 {\scriptstyle ± 5} -> 0 {\scriptstyle ± 0} | Hurt | Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning (NeurIPS 2025 · 0 citations) |
| Train DT on 10 million transitions instead of the smaller dataset | Point maze tasks · combinatorial generalisation | 10 million | No clear effect | Closing the Gap between TD Learning and Supervised Learning - A Generalisation Point of View. (ICLR 2024 · 0 citations) |
| add test-time midpoint planning (three recursions) | AntMaze and Kitchen goal-conditioned tasks · performance | Mixed | Foundation Policies with Hilbert Representations (ICML 2024 · 0 citations) | |
| Train the goal-reducer with loop-removal sampling instead of trajectory sampling | Geometric random graph and four-room gridworld · training loss | Helped | Goal Reduction with Loop-Removal Accelerates RL and Models Human Brain Activity in Goal-Directed Learning (NeurIPS 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 9 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- OGBench pointmaze · 2 of 9 papers
- OGBench antmaze · 2 of 9 papers
- OGBench humanoidmaze · 2 of 9 papers
- OGBench scene · 2 of 9 papers
- OGBench visual-antmaze · 2 of 9 papers
- OGBench visual-cube · 2 of 9 papers
Baselines
- CRL · 5 of 9 papers
- GCBC · 4 of 9 papers
- GCIQL · 4 of 9 papers
- GCIVL · 3 of 9 papers
- QRL · 3 of 9 papers
- HIQL · 3 of 9 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 34 papers; from OpenAlex and the papers' affiliations).
- Google · 4 papers
- University of California, Berkeley · 4 papers
- Seoul National University · 3 papers
- Princeton University · 3 papers
- Pohang University of Science and Technology · 2 papers
- University of California, Los Angeles · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Goal-conditioned policy learning (66 papers)
- Subgoal generation for hierarchical RL (35 papers)
Elsewhere in the topic
- In-context and adaptive RL (202 papers)
- Scalable RL systems and environments (179 papers)
- Probabilistic and autonomous RL (177 papers)
- RL for combinatorial routing (158 papers)
- Learned state and action representations (148 papers)
- Dexterous manipulation transfer (141 papers)
- RL with tree search for discovery (121 papers)
- Inferring decision-making from behavior (93 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).