Research map / Vision-language and multimodal / Human and embodied modeling
Towards Effective Multi-Modal Interchanges in Zero-Resource Sounding Object Localization
Yang Zhao, Chen Zhang, Haifeng Huang, Haoyuan Li, Zhou Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace real query with a random vector in the localization stream (ablation) | TURN · VGGSS | cIoU@0.5: 12.2 -> 34.6 | Improved |
| replace real query with a random vector in the localization stream (ablation) | TURN · MUSIC-Solo | IoU@0.5: 16.9 -> 33.7 | Improved |
| replace real query with a random vector in the localization stream (ablation) | TURN · MUSIC-Solo | AUC: 34.1 -> 45.2 | Improved |
| use RefCOCOg instead of RefCOCO as source domain (AudioCaps intermediate) (ablation) | TURN · VGGSS | cIoU@0.5: 23.13 -> 33.58 | Improved |
| use RefCOCOg instead of RefCOCO+ as source domain (Clotho intermediate) (ablation) | TURN · VGGSS | cIoU@0.5: 22.51 -> 34.57 | Improved |
| use Clotho instead of AudioCaps as intermediate domain (RefCOCOg source) (ablation) | TURN · VGGSS | cIoU@0.5: 33.58 -> 34.57 | Improved |
| use AudioCaps instead of Clotho as intermediate domain (RefCOCOg source) (ablation) | TURN · MUSIC-Solo | IoU@0.5: 28.21 -> 33.68 | Improved |
| remove contrastive loss (largest drop among component ablations) (ablation) | TURN · VGGSS | cIoU@0.5 (Clotho): 34.57 -> 26.44 | Improved |
| remove codebook loss, reconstruction loss or adaptive sampling (one at a time) (ablation) | TURN · VGGSS | cIoU@0.5 (Clotho): 34.57 -> 28.56 | Improved |
| vary total codebook size (too large or too small) (ablation) | TURN codebook · VGGSS | cIoU and AUC | Mixed |
| set codebook group number to 2 and total size to 512 (ablation) | TURN codebook · VGGSS | cIoU and AUC | Improved |
| Self-Cross reconstruction instead of other combinations or none (ablation) | TURN · VGGSS | cIoU and AUC | Improved |
Closest papers by meaning
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.89)
- Object-aware Sound Source Localization via Audio-Visual Scene Understanding (CVPR 2025 · similarity 0.88)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.86)
- Improving Sound Source Localization with Joint Slot Attention on Image and Audio (CVPR 2025 · similarity 0.86)
- Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation (CVPR 2025 · similarity 0.86)
- Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and Language (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
