Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Embodied agent learning
Builds embodied robotic agents using vision-language distillation, affordance exploration, and generalist policies.
93 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Human-AI and goal-conditioned, keyword cluster “navigation · prediction · embodied”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 1 tracked benchmark.
24 papers in 2023–24 and 47 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 8.1% to 12.2%.
Open questions the data points to
- ALFRED test unseen (SR): best went from 36.07 (up to 2024) to 57.82 in 2025 — ThinkBot: Embodied Instruction Following with Thought Chain Reasoning (ICLR 2025 · 1 citation)
- Papers report Prompter on ALFRED Test Unseen (PLWSR) anywhere from 20.76 to 26.22: comparisons against it depend on whose number you take.
- Papers report Prompter on ALFRED Test Unseen (SR) anywhere from 45.72 to 55.46: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Integrating the Cognitive with the Physical: Musical Path Planning for an Improvising Robot (AAAI 2017 · 4 citations)
- 2018 · Gibson Env: Real-World Perception for Embodied Agents (CVPR 2018 · 743 citations)
- 2019 · Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks (CVPR 2019 · 183 citations)
- 2020 · Visual Reaction: Learning to Play Catch With Your Drone (CVPR 2020 · 17 citations)
- 2021 · ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation (NeurIPS 2021 · 127 citations)
- 2022 · Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world (NeurIPS 2022 · 9 citations)
- 2023 · Layout-Aware Dreamer for Embodied Visual Referring Expression Grounding (AAAI 2023 · 12 citations)
- 2024 · Towards Learning a Generalist Model for Embodied Navigation (CVPR 2024 · 63 citations)
- 2025 · From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons (CVPR 2025 · 8 citations)
- 2026 · Cook and Clean Together: Teaching Embodied Agents for Parallel Task Execution (AAAI 2026 · 2 citations)
Earliest papers here
- Integrating the Cognitive with the Physical: Musical Path Planning for an Improvising Robot (AAAI 2017 · 4 citations)
- Gibson Env: Real-World Perception for Embodied Agents (CVPR 2018 · 743 citations)
- Learning to Act Properly: Predicting and Explaining Affordances From Images (CVPR 2018 · 102 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ALFRED test unseen · SR (higher is better)
4 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 36.07 | Egocentric Planning for Scalable Embodied Task Achievement (NeurIPS 2023 · 2 citations) |
| 2025 | 57.82 | ThinkBot: Embodied Instruction Following with Thought Chain Reasoning (ICLR 2025 · 1 citation) |
What has been tried
63 papers here report 596 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 6%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| train jointly on diverse embodied tasks instead of without multi-task learning | CVDN, SOON, R2R, REVERIE, and ScanQA · performance | 5.54 -> 5.91 GP; 28.37 -> 35.44 SR; 21.8 -> 23.3 EM | Helped | Towards Learning a Generalist Model for Embodied Navigation (CVPR 2024 · 63 citations) |
| remove humanoid GPS from the End-to-end RL policy | Social Navigation · Finding Success (S) | 0.97 (± 0.00) -> 0.76 (± 0.02) | Hurt | Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots (ICLR 2024 · 15 citations) |
| Add image + point cloud observations to image observations with text + audio instructions | Real-world tasks · success rate | 56 -> 65 | Helped | Retrieval-Augmented Embodied Agents (CVPR 2024 · 10 citations) |
| Add Stage 2 online RL to SFT training | Habitat Pick · success rate (%) | 60.5 -> 82.5 | Helped | From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons (CVPR 2025 · 8 citations) |
| add object embeddings | Pick & Place (YCB) · success rate (SR) | 70.4 ± 0.4 -> 73.8 ± 4.5 | Helped | Rapid Motor Adaptation for Robotic Manipulator Arms (CVPR 2024 · 8 citations) |
| add MMP (multimodal example retrieval) | ALFRED Test Unseen · SR | 30.35 -> 31.79 | Helped | Multi-Modal Grounded Planning and Efficient Replanning for Learning Embodied Agents with a Few Examples (AAAI 2025 · 7 citations) |
| Use attention-based feature integration instead of average-feature fusion | ShapeNet, 10 base classes, open classes · top-1 accuracy | 50.6 -> 53.3 | Helped | Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations (CVPR 2024 · 6 citations) |
| Initialize two-agent training from the single-agent policy and fine-tune it, rather than training from scratch | Box carrying, two agents · Success Rate (%) | 0 -> 89.54 | Helped | CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics (NeurIPS 2024 · 5 citations) |
| add tactile force feedback | 8 objects · success rate | 82% -> 86% | Helped | Multi-fingered Hand Grasps with Visuo-Tactile Fusion via Multi-Agent Deep Reinforcement Learning (AAAI 2025 · 4 citations) |
| Increase tokens per image observation from 256 to 540 | Bleeding Edge · optimal model-size exponent | 0.49 to 0.62 | Helped | Scaling Laws for Pre-training Agents and World Models (ICML 2025 · 2 citations) |
| Add depth information to the RGB video model | Video generation · visual quality and consistency | Helped | Learning 3D Persistent Embodied World Models (NeurIPS 2025 · 2 citations) | |
| Fine-tune the MetaUrban-train pretrained model on MetaUrban-finetune instead of training directly on MetaUrban-finetune | MetaUrban-unseen distribution · success rate | Helped | MetaUrban: An Embodied AI Simulation Platform for Urban Micromobility (ICLR 2025 · 2 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Prompter | ALFRED Test Unseen | PLWSR | 20.76 – 26.22 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
| Prompter | ALFRED Test Unseen | SR | 45.72 – 55.46 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
| Prompter | ALFRED Test Seen | PLWSR | 25.81 – 31.12 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
| Prompter | ALFRED Test Seen | PLWGC | 30.72 – 36.35 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
| Prompter | ALFRED Test Unseen | PLWGC | 26.22 – 30.09 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
| Prompter | ALFRED Test Seen | SR | 53.23 – 60.86 | 2 papers, e.g. NeurIPS 2023 and ICLR 2025 |
What reviewers will expect
Datasets and baselines named in the 29 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CALVIN · 2 of 29 papers
- ALFRED · 2 of 29 papers
- Meta-World · 2 of 29 papers
Baselines
- GPT-4o · 4 of 29 papers
- LLM-Planner · 3 of 29 papers
- FILM · 2 of 29 papers
- OpenVLA · 2 of 29 papers
- Gemini 2.0 Flash · 2 of 29 papers
- Diffusion Policy · 2 of 29 papers
- Gemini-2.5-Flash · 2 of 29 papers
- GenSim · 2 of 29 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 71 papers; from OpenAlex and the papers' affiliations).
- Meta · 7 papers
- Shanghai AI Laboratory · 6 papers
- Tsinghua University · 5 papers
- University of Washington · 4 papers
- The Chinese University of Hong Kong · 4 papers
- Peking University · 4 papers
- Carnegie Mellon University · 4 papers
- University of Oxford · 3 papers
- Google · 3 papers
- Tongji University · 3 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Interactive multi-agent trajectory prediction (75 papers)
- Vision-language navigation agents (66 papers)
- Data-driven traffic simulation (36 papers)
Elsewhere in the topic
- Human-AI coordination agents (125 papers)
- World-model-based agents (110 papers)
- Imitation and skill learning (100 papers)
- Language-grounded reinforcement agents (96 papers)
- Boundedly rational agent modeling (87 papers)
- Collaborative multi-agent perception (86 papers)
- Intrinsically motivated exploration (86 papers)
- Learning-guided MAPF search (73 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).