Research map / Video / Video segmentation and tracking
Localizing Visual Sounds the Hard Way
Honglie Chen, Weidi Xie, Triantafyllos Afouras, Arsha Nagrani, Andrea Vedaldi, Andrew Zisserman
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hard-negative mining and Tri-map training | Ours · Flickr SoundNet test set, Flickr10k training | cIoU: 0.546 -> 0.582 | Improved |
| Use hard-negative mining and Tri-map training | Ours · VGG-SS test set, VGG-Sound144k training | cIoU: 0.297 -> 0.344 | Improved |
| Add complementary-region negatives without excluding uncertain pixels (ablation) | Ours · Flickr SoundNet test set, VGG-Sound144k training | cIoU: 0.675 -> 0.667 | Hurt |
| Add a Tri-map that ignores uncertain regions during training (ablation) | Ours · Flickr SoundNet test set, VGG-Sound144k training | cIoU: 0.667 -> 0.700 | Improved |
| Set positive threshold to 0.65 and negative threshold to 0.4 (ablation) | Ours with Tri-map · Flickr SoundNet test set, VGG-Sound144k training | cIoU: 0.703 -> 0.719 | Improved |
| Use positive threshold 0.7 and negative threshold 0.3 (ablation) | Ours with Tri-map · Flickr SoundNet test set, VGG-Sound144k training | cIoU: 0.719 -> 0.687 | Hurt |
| Train on VGG-Sound10k instead of Flickr10k | Ours · Flickr SoundNet test set | cIoU: 0.582 -> 0.618 | Improved |
| Train on the full VGG-Sound set rather than VGG-Sound144k | Ours · Flickr SoundNet test set | cIoU: 0.719 -> 0.735 | Improved |
Closest papers by meaning
- Localizing Visual Sounds the Easy Way (ECCV 2022 · similarity 0.94)
- Object-aware Sound Source Localization via Audio-Visual Scene Understanding (CVPR 2025 · similarity 0.89)
- Improving Sound Source Localization with Joint Slot Attention on Image and Audio (CVPR 2025 · similarity 0.89)
- T-VSL: Text-Guided Visual Sound Source Localization in Mixtures (CVPR 2024 · similarity 0.89)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.88)
- Learning to Visually Localize Sound Sources from Mixtures without Prior Source Knowledge (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
