Research map / Vision-language and multimodal / Multimodal LLMs
VisualEchoes: Spatial Image Representation Learning through Echolocation
Ruohan Gao, Changan Chen, Ziad Al-Halah, Carl Schissler, Kristen Grauman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add echo input to RGB depth prediction | RGB2Depth · Replica depth prediction (heldout, case study) | RMS: 0.374 -> 0.346 | Improved |
| use echoes only instead of average depth | Echo2Depth · Replica depth prediction (heldout, case study) | RMS: 1.070 -> 0.713 | Improved |
| pre-train encoder with VisualEchoes instead of training from scratch | RGB2Depth UNet · Replica depth | RMS: 0.360 -> 0.332 | Improved |
| pre-train encoder with VisualEchoes instead of training from scratch | RGB2Depth UNet · NYU-V2 depth (UNet) | RMS: 0.818 -> 0.797 | Improved |
| pre-train encoder with VisualEchoes instead of training from scratch | RGB2Depth UNet · DIODE depth | RMS: 2.352 -> 2.223 | Improved |
| pre-train with ImageNet classification instead of VisualEchoes | RGB2Depth UNet · Replica depth | RMS: 0.332 -> 0.356 | Hurt |
| use 4-class orientation prediction instead of 2-class (ablation) | VisualEchoes · Replica depth | RMS: 0.340 -> 0.332 | Improved |
| use orientation prediction instead of binary echo-image matching (ablation) | VisualEchoes · Replica depth | RMS: 0.345 -> 0.332 | Improved |
| pre-train ResNet-50 with VisualEchoes instead of scratch | depth network · NYU-V2 depth | RMS: 0.804 -> 0.683 | Improved |
| pre-train with ImageNet instead of VisualEchoes | depth network · NYU-V2 depth | RMS: 0.683 -> 0.555 | Hurt |
| pre-train PSPNet with VisualEchoes instead of scratch | PSPNet · NYU-V2 surface normals | mean angle distance: 26.3 -> 22.9 | Improved |
| pre-train navigation CNN with VisualEchoes instead of scratch | PointGoal agent · Replica navigation | SPL: 0.830 -> 0.856 | Improved |
Later papers that compared with this method
1 later accepted paper reports VisualEchoes as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Beyond Image to Depth: Improving Depth Prediction Using Echoes (CVPR 2021 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.86)
- SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing (NeurIPS 2025 · similarity 0.86)
- RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics (CVPR 2025 · similarity 0.86)
- Spatial Understanding from Videos: Structured Prompts Meet Simulation Data (NeurIPS 2025 · similarity 0.85)
- Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models (CVPR 2026 · similarity 0.85)
- BOP-ASK: Object-Interaction Reasoning for Vision-Language Models (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
