Research map / Vision-language and multimodal / Human and embodied modeling
Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cyclic feedback from separation to grounding sample mining (ablation) | CoL / CCoL · MIT MUSIC, single-sound grounding | accuracy (%): 72.0 -> 84.5 | Helped |
| add cyclic feedback from separation to grounding sample mining (ablation) | CoL / CCoL · MIT MUSIC, mixed-sound grounding | accuracy (%): 58.2 -> 75.9 | Helped |
| jointly learn grounding and separation without cyclic feedback (ablation) | Grounding only · MIT MUSIC, single-sound grounding | accuracy (%): 72.0 -> 67.0 | Hurt |
| jointly learn grounding and separation without cyclic feedback (ablation) | Grounding only · MIT MUSIC, mixed-sound grounding | accuracy (%): 61.4 -> 58.2 | Hurt |
| add sounding-object-aware separation guided by grounding (ablation) | audio-visual sound separation · MIT MUSIC | SDR: 4.20 -> 6.50 | Helped |
| add cyclic feedback between grounding and separation (ablation) | CoL · MIT MUSIC | SDR: 6.50 -> 7.27 | Helped |
| add cyclic feedback between grounding and separation (ablation) | CoL · MIT MUSIC | SIR: 11.81 -> 12.77 | Helped |
| use estimated grounding labels instead of ground-truth labels | Oracle separation model · MIT MUSIC | SDR: 7.71 -> 7.27 | Hurt |
| use estimated grounding labels instead of ground-truth labels | Oracle separation model · MIT MUSIC | SIR: 11.42 -> 12.77 | Helped |
| add cyclic feedback between grounding and separation (ablation) | CoL · MIT MUSIC with silent objects | SDR: 14.78 -> 91.07 | Helped |
| add cyclic feedback between grounding and separation (ablation) | CoL · MIT MUSIC with silent objects | SIR: 15.68 -> 82.82 | Helped |
| use cyclic co-learning rather than CoL (ablation) | CoL · MIT MUSIC, solo and duet videos | SDR: 5.79 -> 6.46 | Helped |
Closest papers by meaning
- Cyclic Learning for Binaural Audio Generation and Localization (CVPR 2024 · similarity 0.87)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.87)
- Continual Audio-Visual Sound Separation (NeurIPS 2024 · similarity 0.87)
- Sounding that Object: Interactive Object-Aware Image to Audio Generation (ICML 2025 · similarity 0.87)
- Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation (CVPR 2025 · similarity 0.85)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.