AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Multiple Sound Sources Localization from Coarse to Fine

Rui Qian, Di Hu, Heinrich Dinkel, Mengyue Wu, Ning Xu, Weiyao Lin

ECCV 2020 · Vision-language and multimodal › Cross-modal learning · approach: Audio-visual source localization · 163 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add fine-grained audiovisual alignment over multi-task learning (ablation)multi-task baseline · AudioSet level-1cIoU_class: 20.6 -> 32.8Improved
add fine-grained audiovisual alignment over multi-task learning (ablation)multi-task baseline · AudioSet level-2cIoU_class: 2.37 -> 6.16Improved
add fine-grained audiovisual alignment over multi-task learning (ablation)multi-task baseline · AudioSet level-3cIoU_class@0.3: 10.5 -> 21.1Improved
add fine-grained alignment and classification over video-level AVC (ablation)AVC baseline · AudioSet level-3cIoU_class@0.3: 5.3 -> 21.1Improved
add classification to AVC (multi-task) when sources are many (ablation)AVC baseline · AudioSet level-3cIoU_class@0.3: 5.3 -> 10.5Improved
add classification to AVC (multi-task) when sources are few (ablation)AVC baseline · AudioSet level-1cIoU_class: 24.8 -> 20.6Hurt
add fine-grained alignment over AVC (ablation)AVC baseline · AudioSet level-2AUC: 23.6 -> 23.9No clear effect
increase training data from 10K to 20K pairsOurs · SoundNet-FlickrcIoU@0.5: 52.2 -> 53.8Improved
use unconstrained-video training with alignment instead of single-source CAVLOurs 10K vs CAVL · SoundNet-FlickrcIoU@0.5: 50.0 -> 52.2Improved
use visual representation instead of Grad-CAM weights as separation guidance (ablation)Ours · MUSIC single-sourceSDR: -2.78 -> 6.53Improved
use visual representation instead of audio representation as separation guidance (ablation)Ours · MUSIC single-sourceSDR: -1.16 -> 6.53Improved
train with duet videos in addition to solo videosOurs Visual · MUSICSDR: 6.53 -> 6.57No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy