Research map / Vision-language and multimodal / Robot action and navigation
Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MCoCoNav, a modular zero-shot multi-robot semantic navigation planner that uses cross-image multimodal chain-of-thought with a locally deployed quantized GLM-4V-9B and a shared global semantic map as the communication channel.
Problem
Multi-robot object-goal semantic navigation in unseen indoor environments, where centralized planners scale poorly and decentralized planners incur high communication and time costs; prior LLM/VLM approaches rely on remote large models.
Approach
Robots share a global semantic map as the communication bridge, and a local VLM scores exploration value via the probability of "Yes" over multimodal CoT prompts combining the scene view, detections, and the annotated map; history-node scores let robots choose between frontier exploration and revisiting history nodes.
Key results
- 2-robot MCoCoNav with local GLM-4V-9B (INT4) reaches SPL 0.387 and SR 0.716 on HM3D_v0.2, versus 0.331/0.661 for Co-NavGPT with GPT-3.5.
- On MP3D, MCoCoNav achieves SPL 0.334 and SR 0.568 versus 0.281/0.435 for Random Sampling.
- Single-robot MCoCoNav reaches SPL 0.297 and SR 0.634 on HM3D, outperforming OpenFMNav (0.238/0.529).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MCoCoNav instead of Co-NavGPT | 2-robot navigation · HM3D_v0.2 | SPL: 0.331 -> 0.387; +5.6% SPL | Helped |
| Use MCoCoNav instead of Co-NavGPT | 2-robot navigation · HM3D_v0.2 | SR: 0.661 -> 0.716; +5.5% SR | Helped |
| Use MCoCoNav instead of Random Sampling | 2-robot navigation · MP3D | SPL: +5.3% SPL | Helped |
| Use MCoCoNav instead of Random Sampling | 2-robot navigation · MP3D | SR: +13.3% SR | Helped |
| Replace predicted semantic segmentation with ground-truth segmentation | MCoCoNav · HM3D_v0.2 (2 robots) | SR: +11.5% SR | Helped |
| Add Perception multimodal CoT to Perception + Decision (ablation) | MCoCoNav · HM3D_v0.2 | SR: 0.677 -> 0.698 | Helped |
| Add Logical Analysis to Perception + PCoT + Judgment + Decision (ablation) | MCoCoNav · HM3D_v0.2 | SPL: 0.381 -> 0.387 | Helped |
| Add Logical Analysis to Perception + PCoT + Judgment + Decision (ablation) | MCoCoNav · HM3D_v0.2 | SR: 0.707 -> 0.716 | Helped |
| Increase the robot count from 1 to 2 | MCoCoNav · HM3D_v0.2 | SPL: 0.297 -> 0.387 | Helped |
Surprising findings
- Co-NavGPT drops from SPL 0.331 with GPT-3.5 to 0.316 with the local GLM-4V, showing a remote-to-local VLM performance gap for prior methods.
- Adding a third robot raises SR only from 0.716 to 0.720 with predicted segmentation while time per episode grows substantially (883s to 1110s).
Limitations
- Detection failures from inaccurate semantic segmentation and 3D scan quality dominate failures, and grow with more robots.
- Performance depends on a quantized, small local VLM; the approach was not compared against stronger VLMs beyond GLM-4V and GPT-3.5 on Co-NavGPT.
Experimental setup
Datasets: HM3D_v0.2 MP3D
Benchmarks: HM3D_v0.2 MP3D
Metrics: SPL SR time per episode detection trap count
Baselines: Greedy Cost-Utility Random Sampling Multi-SemExp Co-NavGPT SemExp L3MVN Pixel-Nav ESC OpenFMNav Frontier Based
Builds on: Co-NavGPT Multi-SemExp Fast Marching Method chain-of-thought prompting
Closest papers by meaning
- Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation (NeurIPS 2025 · similarity 0.89)
- FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-and-Language Navigation (CVPR 2026 · similarity 0.88)
- ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination (ICLR 2025 · similarity 0.87)
- Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation (AAAI 2026 · similarity 0.87)
- Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.87)
- APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.