Research map / Label-efficient and robust learning / Self-supervised and contrastive
Self-Supervised Visual Acoustic Matching
Arjun Somayazulu, Changan Chen, Kristen Grauman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual input (vs non-visual LeMARA) (ablation) | LeMARA · SoundSpaces-Speech unseen | RTE (s): 0.152 -> 0.079 | Improved |
| add visual input (vs non-visual LeMARA) (ablation) | LeMARA · AVSpeech-Rooms unseen | RTE (s): 0.137 -> 0.071 | Improved |
| add visual input (vs non-visual LeMARA) (ablation) | LeMARA · LibriSpeech | RTE (s): 0.223 -> 0.210 | Improved |
| use acoustic residue metric instead of SRMR alone as training objective (ablation) | LeMARA · LibriSpeech | RTE: 0.2308 -> 0.2156 | Improved |
| combine acoustic residue with SRMR (ablation) | LeMARA · LibriSpeech | RTE: 0.2156 -> 0.2123 | Improved |
| add shortcut training with SoundSpaces de-biaser (ablation) | LeMARA · LibriSpeech | RTE: 0.2123 -> 0.2100 | Improved |
| LeMARA training objective vs ViGAS with same reverberator architecture | LeMARA · SoundSpaces-Speech unseen | RTE (s): 0.108 -> 0.079 | Improved |
| LeMARA vs AViTAR on STFT error | LeMARA · AVSpeech-Rooms unseen | STFT: 2.894 -> 6.298 | Hurt |
| LeMARA vs AViTAR on logSTFT | LeMARA · AVSpeech-Rooms unseen | logSTFT: 1.290 -> 0.571 | Improved |
| LeMARA vs AViTAR in human room identification | LeMARA · Human study | accuracy (%): 34.7 -> 46.1 | Improved |
| de-biasing vs dereverberation of audio | de-biaser · AVSpeech-Rooms | SRMR: 8.99 -> 13.14 | Improved |
| de-biasing vs dereverberation of audio | de-biaser · SoundSpaces-Speech | RT60: 0.06 -> 0.04 | Improved |
Closest papers by meaning
- Visual Acoustic Matching (CVPR 2022 · similarity 0.94)
- Language-Guided Audio-Visual Source Separation via Trimodal Consistency (CVPR 2023 · similarity 0.89)
- Mutual Learning for Acoustic Matching and Dereverberation via Visual Scene-driven Diffusion (ECCV 2024 · similarity 0.89)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.87)
- SoundBrush: Sound as a Brush for Visual Scene Editing (AAAI 2025 · similarity 0.86)
- Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
