Research map / Vision-language and multimodal / Cross-modal learning
Semantic Audio-Visual Navigation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SAVi's goal descriptor and multimodal transformer memory | SAVi · SoundSpaces Matterport3D, heard sounds | SPL: 16.8 -> 24.0 | Helped |
| use SAVi's goal descriptor and multimodal transformer memory | SAVi · SoundSpaces Matterport3D, unheard sounds | SPL: 13.4 -> 17.2 | Helped |
| use SAVi with a goal descriptor for navigation despite unheard distractors | semantic audio-visual navigation · Unheard sounds with unheard distractors | Success: 4.2 -> 11.8 | Helped |
| combine location and category descriptors rather than use category alone (ablation) | SAVi · Unheard sounds | Success: 20.5 -> 24.8 | Helped |
| combine location and category descriptors rather than use location alone (ablation) | SAVi · Unheard sounds | Success: 23.9 -> 24.8 | Helped |
| add temporal aggregation of current and previous goal estimates (ablation) | SAVi · Unheard sounds | SPL: 14.3 -> 17.2 | Helped |
| replace the transformer policy with an RNN+MLP (ablation) | SAVi · Unheard sounds | SPL: 17.2 -> 15.4 | Hurt |
| train the location predictor on-policy rather than off-policy (ablation) | SAVi · Unheard sounds | Success: 4.8% success rate | Helped |
| provide the true goal category and location instead of predicted descriptors (ablation) | SAVi · Unheard sounds | SPL: 24% -> 65% | Helped |
| replace the predicted category with a random category label (ablation) | SAVi · Unheard sounds | Success | Hurt |
Closest papers by meaning
- Towards Audio-Visual Navigation in Noisy Environments: A Large-Scale Benchmark Dataset and an Architecture Considering Multiple Sound-Sources (AAAI 2025 · similarity 0.91)
- RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation (NeurIPS 2026 · similarity 0.89)
- Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs (NeurIPS 2026 · similarity 0.89)
- RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation (CVPR 2024 · similarity 0.88)
- SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing (NeurIPS 2025 · similarity 0.87)
- CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.