Research map / Vision-language and multimodal / Cross-modal learning
Multiple Sound Sources Localization from Coarse to Fine
Rui Qian, Di Hu, Heinrich Dinkel, Mengyue Wu, Ning Xu, Weiyao Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add fine-grained audiovisual alignment over multi-task learning (ablation) | multi-task baseline · AudioSet level-1 | cIoU_class: 20.6 -> 32.8 | Improved |
| add fine-grained audiovisual alignment over multi-task learning (ablation) | multi-task baseline · AudioSet level-2 | cIoU_class: 2.37 -> 6.16 | Improved |
| add fine-grained audiovisual alignment over multi-task learning (ablation) | multi-task baseline · AudioSet level-3 | cIoU_class@0.3: 10.5 -> 21.1 | Improved |
| add fine-grained alignment and classification over video-level AVC (ablation) | AVC baseline · AudioSet level-3 | cIoU_class@0.3: 5.3 -> 21.1 | Improved |
| add classification to AVC (multi-task) when sources are many (ablation) | AVC baseline · AudioSet level-3 | cIoU_class@0.3: 5.3 -> 10.5 | Improved |
| add classification to AVC (multi-task) when sources are few (ablation) | AVC baseline · AudioSet level-1 | cIoU_class: 24.8 -> 20.6 | Hurt |
| add fine-grained alignment over AVC (ablation) | AVC baseline · AudioSet level-2 | AUC: 23.6 -> 23.9 | No clear effect |
| increase training data from 10K to 20K pairs | Ours · SoundNet-Flickr | cIoU@0.5: 52.2 -> 53.8 | Improved |
| use unconstrained-video training with alignment instead of single-source CAVL | Ours 10K vs CAVL · SoundNet-Flickr | cIoU@0.5: 50.0 -> 52.2 | Improved |
| use visual representation instead of Grad-CAM weights as separation guidance (ablation) | Ours · MUSIC single-source | SDR: -2.78 -> 6.53 | Improved |
| use visual representation instead of audio representation as separation guidance (ablation) | Ours · MUSIC single-source | SDR: -1.16 -> 6.53 | Improved |
| train with duet videos in addition to solo videos | Ours Visual · MUSIC | SDR: 6.53 -> 6.57 | No clear effect |
Closest papers by meaning
- Object-aware Sound Source Localization via Audio-Visual Scene Understanding (CVPR 2025 · similarity 0.90)
- T-VSL: Text-Guided Visual Sound Source Localization in Mixtures (CVPR 2024 · similarity 0.89)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.88)
- Learning to Visually Localize Sound Sources from Mixtures without Prior Source Knowledge (CVPR 2024 · similarity 0.88)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.88)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
