Research map / Vision-language and multimodal / Cross-modal learning
SoundSpaces: Audio-Visual Navigation in 3D Environments
Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, and Kristen Grauman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio (AudioPointGoal vs PointGoal) | Blind agent · Replica | SPL: 0.480 -> 0.681 | Improved |
| add audio (AudioPointGoal vs PointGoal) | RGB agent · Replica | SPL: 0.521 -> 0.632 | Improved |
| add audio (AudioPointGoal vs PointGoal) | Depth agent · Replica | SPL: 0.601 -> 0.709 | Improved |
| add audio (AudioPointGoal vs PointGoal) | Blind agent · Matterport3D | SPL: 0.426 -> 0.473 | Improved |
| add audio (AudioPointGoal vs PointGoal) | RGB agent · Matterport3D | SPL: 0.466 -> 0.521 | Improved |
| add audio (AudioPointGoal vs PointGoal) | Depth agent · Matterport3D | SPL: 0.541 -> 0.581 | Improved |
| use spectrograms instead of intensity-only audio input (ablation) | AudioGoal Depth · Replica | SPL: 0.291 -> 0.756 | Improved |
| use spectrograms instead of intensity-only audio input (ablation) | AudioGoal Depth · Matterport3D | SPL: 0.014 -> 0.552 | Improved |
| add 40dB SNR microphone noise | AudioGoal Depth · Replica | SPL: 0.756 -> 0.753 | No clear effect |
| add 40dB SNR microphone noise | AudioGoal Depth · Matterport3D | SPL: 0.552 -> 0.550 | No clear effect |
| test on unheard instead of same sounds | AudioGoal Depth · Replica | SPL: 0.756 -> 0.454 | Hurt |
| test on unheard instead of same sounds | AudioPointGoal Depth · Replica | SPL: 0.709 -> 0.707 | No clear effect |
Closest papers by meaning
- Towards Audio-Visual Navigation in Noisy Environments: A Large-Scale Benchmark Dataset and an Architecture Considering Multiple Sound-Sources (AAAI 2025 · similarity 0.90)
- CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments (AAAI 2024 · similarity 0.89)
- Audible World Models: Spatially Aware Sound Generation for 3D Worlds (NeurIPS 2026 · similarity 0.89)
- RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation (CVPR 2024 · similarity 0.88)
- Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator (NeurIPS 2026 · similarity 0.87)
- SoundVista: Novel-View Ambient Sound Synthesis via Visual-Acoustic Binding (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
