Research map / Video / Video segmentation and tracking
Localize to Binauralize: Audio Spatialization From Visual Sound Source Localization
Kranthi Kumar Rachavarapu, Aakanksha, Vignesh Sundaresha, A. N. Rajagopalan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ILD and IPD feature extraction before the localization network (ablation) | L2BNet-WSS (10%) · FAIR-Play | STFT / Envelope distance: 1.143/0.152 -> 1.121/0.151 | Improved |
| add attention (ablation) | L2BNet-WSS (10%) · FAIR-Play | STFT / Envelope distance: 1.162/0.155 -> 1.121/0.151 | Improved |
| use ILD & ITD input instead of monaural audio for audio-only localization | AL network · FAIR-Play | F-measure: 0.096 -> 0.394 | Improved |
| use binaural audio input instead of monaural audio for localization | AL network · FAIR-Play | MAE: 0.1766 -> 0.0281 | Improved |
| use ILD & ITD instead of raw binaural audio for localization | AL network · FAIR-Play | F-measure: 0.380 -> 0.394 | Improved |
| add weak localization supervision to 10% binaural supervision | Ours-SG (10%) · FAIR-Play | STFT distance: 1.188 -> 1.121 | Improved |
| add weak localization supervision to 10% binaural supervision | Ours-SG (10%) · YT-Music | STFT distance: 2.156 -> 1.908 | Improved |
| add weak localization supervision to 30% binaural supervision | Ours-SG (30%) · FAIR-Play | STFT distance: 1.109 -> 1.028 | Improved |
| add weak localization supervision to 30% binaural supervision | Ours-SG (30%) · YT-Music | STFT distance: 1.855 -> 1.816 | Improved |
| raise explicit binaural supervision above 30% with localization task (ablation) | L2BNet-WSS · FAIR-Play | STFT distance | No clear effect |
| use L2BNet-WSS (10%) instead of fully supervised Mono2Binaural | L2BNet-WSS (10%) · FAIR-Play | STFT distance: 0.951 -> 1.121 | Hurt |
Closest papers by meaning
- Binaural Audio-Visual Localization (AAAI 2021 · similarity 0.93)
- Cyclic Learning for Binaural Audio Generation and Localization (CVPR 2024 · similarity 0.92)
- Supervising Sound Localization by In-the-wild Egomotion (CVPR 2025 · similarity 0.90)
- Seeing Sound, Hearing Sight: Uncovering Modality Bias and Conflict of AI models in Sound Localization (NeurIPS 2025 · similarity 0.87)
- BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Models (ICML 2025 · similarity 0.87)
- Learning Spatial Features from Audio-Visual Correspondence in Egocentric Videos (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
