Research map / Vision-language and multimodal / Cross-modal learning
Move2Hear: Active Audio-Visual Source Separation
Sagnik Majumder, Ziad Al-Halah, Kristen Grauman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add acoustic memory refiner f^R (ablation) | Move2Hear · Far-target (heard) | SI-SDR: 2.64 -> 3.50 | Improved |
| add acoustic memory refiner f^R (ablation) | Move2Hear · Far-target (unheard) | SI-SDR: 1.57 -> 2.33 | Improved |
| add visual input V_t (ablation) | Move2Hear · Far-target (heard) | SI-SDR: 3.32 -> 3.50 | Improved |
| add navigation policy pi^N (composite policy) (ablation) | Move2Hear · Far-target (heard) | SI-SDR: 2.64 -> 3.50 | Improved |
| add quality policy pi^Q (ablation) | Move2Hear · Far-target (unheard) | SI-SDR: 1.99 -> 2.33 | Improved |
| use Move2Hear instead of standing in place | Move2Hear · Near-target (unheard) | SI-SDR: 2.40 -> 3.20 | Improved |
| use Move2Hear instead of AV Navigator | Move2Hear · Far-target (unheard) | SI-SDR: 0.72 -> 2.33 | Improved |
| train navigation for audio-goal (AV Navigator) rather than novelty exploration | AV Navigator · Far-target (heard) | SI-SDR: 1.74 -> 1.46 | Hurt |
| use SOTA passive separator MMDenseNet backbone | Move2Hear · Far-target (heard) | SI-SDR: 3.45 -> 6.84 | Improved |
| use SNR as RL reward instead of separation error (ablation) | Move2Hear · Near-target | SI-SDR: 5.1% and 2.2% degradation in SI-SDR | Hurt |
| switch policies very early or never (T^N=100) (ablation) | Move2Hear composite policy · Far-target (heard, validation) | SI-SDR | Hurt |
Later papers that compared with this method
1 later accepted paper reports Move2Hear as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Active Audio-Visual Separation of Dynamic Sound Sources (ECCV 2022 · better on 4 of 4 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments (AAAI 2024 · similarity 0.85)
- Towards Audio-Visual Navigation in Noisy Environments: A Large-Scale Benchmark Dataset and an Architecture Considering Multiple Sound-Sources (AAAI 2025 · similarity 0.85)
- Continual Audio-Visual Sound Separation (NeurIPS 2024 · similarity 0.84)
- PrismAudio: Decomposed Chain-of-Thought and Multi-dimensional Rewards for Video-to-Audio Generation (ICLR 2026 · similarity 0.84)
- Seeing Sound, Hearing Sight: Uncovering Modality Bias and Conflict of AI models in Sound Localization (NeurIPS 2025 · similarity 0.84)
- RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
