AI Paper Atlas

Research map / Video / Video segmentation and tracking

Telling Left From Right: Learning Spatial Correspondence of Sight and Sound

Karren Yang, Bryan Russell, Justin Salamon

CVPR 2020 · Video › Video segmentation and tracking · approach: Sign language and physiological video analysis · 78 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add a mono-audio stream to the flipping-task model (ablation)pretext classifier · YouTube-ASMR and FAIR-Playaccuracy: 80.1 -> 81.3; 93.6 -> 96.3Improved
replace ResNet-18 with MCx motion features (ablation)pretext classifier · YouTube-ASMR and FAIR-Playaccuracy: 80.1 -> 80.4; 93.6 -> 71.3Mixed
add GCC-Phat features (ablation)pretext classifier · YouTube-ASMR and FAIR-Playaccuracy: 80.1 -> 80.1; 93.6 -> 94.1No clear effect
use the proposed model instead of the strongest listed baselineaudio-visual face localization · YouTube-ASMR testAP50: 27.9 -> 43.4Improved
initialize the localization model with flipping-task pretrained weights (ablation)audio-visual face localization · YouTube-ASMR localization testAP50: 23.9 -> 34.9Improved
use flipping-task visual features rather than ImageNet supervised featuresaudio upmixing · YouTube-ASMRL1 error: 0.0858 -> 0.0853Improved
use flipping-task visual features rather than ImageNet supervised featuresaudio upmixing · FAIR-PlayL1 error: 0.403 -> 0.401Improved
combine flipping-task features with ImageNet features (ablation)source separation · YouTube-ASMRL1 error: 0.0885 -> 0.0863Improved
combine flipping-task features with ImageNet features (ablation)source separation · FAIR-PlayL1 error: 0.362 -> 0.350Improved
initialize the DOA model with self-supervised 360-degree audio-visual pretrained weightsDOA estimation · Tau Spatial Soundmean error: 20.5 -> 17.5Improved
use self-supervised embeddings rather than random embeddings for one-shot SVMDOA estimation · Tau Spatial Sound one-shotmean error: 78.6 -> 29.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy