Research map / Vision-language and multimodal
Robot action and navigation: research map
841 papers on Robot action and navigation in Vision-language and multimodal, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 6 approaches. The busiest year so far is 2026.
What it is
Models that turn what a robot sees and an instruction in words into actions. Vision-language-action (VLA) models extend multimodal LLMs to output robot actions; vision-language navigation (VLN) agents follow route instructions in buildings.
Terms you will see on this page:
- VLA
- vision-language-action model, mapping images and instructions to robot actions.
- VLN
- vision-language navigation, following natural-language directions in a 3D environment.
- Embodied AI
- AI that perceives and acts in a physical or simulated world.
Start reading with the most cited papers here:
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments (CVPR 2018 · 1,195 citations)
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation (CVPR 2019 · 548 citations)
Within Vision-language and multimodal, its share grew from 4.8% in 2023–24 to 10.4% in 2025–26 (76 → 497 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Robot action and navigation in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Vision-language-action foundation models rising: 9% → 52% of the topic (7 → 257 papers)
- Pretraining and data augmentation for VLN falling: 24% → 10% of the topic (18 → 51 papers)
- Reasoning-enhanced vision-language navigation falling: 16% → 4% of the topic (12 → 21 papers)
Papers from 2025–2026 that are already cited most:
- CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models (CVPR 2025 · 58 citations)
- SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment (CVPR 2025 · 28 citations)
- RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete (CVPR 2025 · 23 citations)
- AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · 16 citations)
Approaches and key papers
Vision-language-action models · 627 papers
Keywords: vla, language action, robotic
Approaches in this cluster:
- Vision-language-action foundation models (323 papers)
Trains and benchmarks end-to-end VLA policies, with reasoning, unified representations and efficient token caching. - Memory and subgoal-based manipulation (143 papers)
Improves robot manipulation with memory, behavioral phasing, subgoal images and skill decomposition. - Embodied multimodal LLM agents (98 papers)
Applies pretrained multimodal LLMs and embodied chain-of-thought to interactive agents, with benchmarks. - Multimodal prompt-driven manipulation (63 papers)
Conditions robot policies on multimodal prompts and object-centric or programmatic plans from language instructions.
Most cited and most cited since 2024:
- ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks (CVPR 2020 · 533 citations)
- PaLM-E: An Embodied Multimodal Language Model (ICML 2023 · 349 citations)
- ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation (CVPR 2024 · 82 citations)
- Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision Language Audio and Action (CVPR 2024 · 77 citations)
Vision-language navigation · 214 papers
Keywords: language navigation, vln, agent
Approaches in this cluster:
- Pretraining and data augmentation for VLN (138 papers)
Improves navigation agents through pretraining, synthetic instructions, imitation learning and exploration with scene objects. - Reasoning-enhanced vision-language navigation (76 papers)
Adds auxiliary tasks, knowledge, concept learning and structured state modeling to navigation policies.
Most cited and most cited since 2024:
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments (CVPR 2018 · 1,195 citations)
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation (CVPR 2019 · 548 citations)
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models (AAAI 2024 · 184 citations)
- Vision-and-Language Navigation via Causal Learning (CVPR 2024 · 40 citations)
Related topics in Vision-language and multimodal
- Human and embodied modeling (2,095)
- Visual question answering (711)
- Multimodal LLMs (3,132)
- CLIP and zero-shot (1,294)
- Cross-modal learning (1,766)
