Research map / Vision-language and multimodal / Cross-modal learning
Localizing Visual Sounds the Easy Way
Shentong Mo, Pedro Morgado
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add L1 object-guided localization to audio-visual localization maps (ablation) | EZ-VSL · Flickr SoundNet test (ablation, trained on VGG-Sound Full) | CIoU: 78.31 -> 83.94 | Improved |
| add L1 object-guided localization to audio-visual localization maps (ablation) | EZ-VSL · VGG-SS test (ablation, trained on VGG-Sound Full) | CIoU: 35.96 -> 39.34 | Improved |
| add audio-visual localization to L1-OGL (ablation) | EZ-VSL · VGG-SS test (ablation, trained on VGG-Sound Full) | CIoU: 36.77 -> 39.34 | Improved |
| add CLS-based object prior to audio-visual localization (ablation) | EZ-VSL · Flickr SoundNet test (ablation, trained on VGG-Sound Full) | CIoU: 78.31 -> 81.93 | Improved |
| use L1 activation prior instead of classification-based prior (ablation) | EZ-VSL · Flickr SoundNet test (ablation, trained on VGG-Sound Full) | CIoU: 81.93 -> 83.94 | Improved |
| use max pooling of similarity over locations instead of average pooling (ablation) | EZ-VSL · Flickr SoundNet test (ablation, trained on VGG-Sound Full) | CIoU: 33.33 -> 78.31 | Improved |
| use max pooling of similarity over locations instead of average pooling (ablation) | EZ-VSL · VGG-SS test (ablation, trained on VGG-Sound Full) | CIoU: 6.03 -> 35.96 | Improved |
| match audio to localized visual features instead of a pooled global visual feature (ablation) | EZ-VSL · Flickr SoundNet test (ablation, trained on VGG-Sound Full) | CIoU: 49.40 -> 78.31 | Improved |
| use non-linear instead of linear projections to the shared space (ablation) | EZ-VSL · Flickr SoundNet test (ablation, trained on VGG-Sound Full) | CIoU | No clear effect |
| train with multiple-instance contrastive loss instead of prior methods (LVS) | LVS · Flickr SoundNet test (trained on Flickr 10k) | CIoU: 58.20 -> 81.93 | Improved |
| train with multiple-instance contrastive loss instead of LVS | LVS · VGG-SS open set (unheard 110 classes, trained on 70k) | CIoU: 26.30 -> 39.57 | Improved |
Closest papers by meaning
- A Closer Look at Weakly-Supervised Audio-Visual Source Localization (NeurIPS 2022 · similarity 0.95)
- Localizing Visual Sounds the Hard Way (CVPR 2021 · similarity 0.94)
- Object-aware Sound Source Localization via Audio-Visual Scene Understanding (CVPR 2025 · similarity 0.93)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.92)
- T-VSL: Text-Guided Visual Sound Source Localization in Mixtures (CVPR 2024 · similarity 0.91)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
