AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Visual Scene Graphs for Audio Source Separation

Moitreya Chatterjee, Jonathan Le Roux, Narendra Ahuja, Anoop Cherian

ICCV 2021 · Vision-language and multimodal › Cross-modal learning · approach: Audio-visual source localization · 31 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the mutual-orthogonality loss on sub-graph embeddings (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 7.37 -> 8.75, SIR 13.28 -> 14.06, SAR 11.56 -> 12.95Improved
add the consistency (multi-label classification) loss (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 6.4 -> 8.75, SIR 11.2 -> 14.06, SAR 11.7 -> 12.95Improved
add the co-separation loss (ablation)AVSGS · ASIW testSDR / SIR (dB): SDR 1.1 -> 8.75, SIR 1.3 -> 14.06Improved
add the co-separation loss (effect on artifacts) (ablation)AVSGS · ASIW testSAR (dB): SAR 13.8 -> 12.95Hurt
increase the number of principal objects per video from 2 to 3 (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 8.75 -> 8.4, SIR 14.06 -> 13.5, SAR 12.95 -> 12.2No clear effect
add skip connections in the audio separator network (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 2.8 -> 8.75, SIR 4.6 -> 14.06, SAR 11.3 -> 12.95Improved
add the graph attention convolution (GATConv) on the scene graph (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 6.5 -> 8.75, SIR 11.6 -> 14.06, SAR 11.8 -> 12.95Improved
add the edge convolution (EdgeConv) on the scene graph (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 6.2 -> 8.75, SIR 10.1 -> 14.06, SAR 13.2 -> 12.95Improved
add the GRU for autoregressive sub-graph embeddings (ablation)AVSGS · ASIW testSDR / SIR / SAR (dB): SDR 6.49 -> 8.75, SIR 12.33 -> 14.06, SAR 10.64 -> 12.95Improved
build the scene graph from multiple frames and use more context nodes at test time (vs single frame) (ablation)AVSGS · ASIW testseparation performanceImproved
replace visual-context-limited baselines by scene-graph conditioning (AVSGS vs closest competitor) on MUSICvisually guided separation · MUSIC testSDR / SIR (dB): around 1.3 dB on SDR and 1.6 dB on SIRImproved
replace closest competitor with AVSGS on ASIWvisually guided separation · ASIW testSDR / SIR (dB): around 1.2 dB on SDR and 2 dB on SIRImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy