Research map / Image generation / GANs
High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling
Chao Huang, Susan Liang, Yapeng Tian, Anurag Kumar, Chenliang Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Time-Frequency and Time attention blocks to CA blocks instead of ResNet only (ablation) | DAVIS · MUSIC val | SDR: 9.03 -> 11.88 | Improved |
| add audio-visual Feature Interaction Module instead of plain concatenation (ablation) | DAVIS · MUSIC val | SDR: 10.85 -> 11.88 | Improved |
| use both local and global interaction in FIM instead of global only (ablation) | DAVIS · MUSIC val | SDR: 11.23 -> 11.88 | Improved |
| add silence mask-guided sampling with threshold 0.002 (ablation) | DAVIS · MUSIC test | SDR: 11.53 -> 11.61 | Improved |
| use a high silence threshold 0.01 in mask-guided sampling (ablation) | DAVIS · MUSIC test | SDR: 11.53 -> 11.15 | Hurt |
| train with L1 loss instead of L2 (ablation) | DAVIS · MUSIC test | SDR: 10.84 -> 11.61 | Improved |
| train with L1 loss instead of L2 (ablation) | DAVIS · AVE test | SDR: 4.53 -> 4.86 | Improved |
| replace DDPM with Flow Matching | DAVIS · MUSIC test | SDR: 11.61 -> 12.01 | Improved |
| replace DDPM with Flow Matching | DAVIS · AVE test | SDR: 4.86 -> 5.66 | Improved |
| use Flow Matching with 2 Euler steps instead of DDPM with 15 DDIM steps | DAVIS · Inference latency (RTX 4090) | latency (s): 1.171 -> 0.156 | Improved |
| replace global CLIP embeddings with GroundingDINO localized embeddings at test time | DAVIS-Flow · MUSIC test | SDR: 12.01 -> 11.75 | No clear effect |
| replace global CLIP embeddings with GroundingDINO localized embeddings at test time | DAVIS-Flow · AVE test | SDR: 5.66 -> 5.29 | No clear effect |
Closest papers by meaning
- Multi-Source Diffusion Models for Simultaneous Music Generation and Separation (ICLR 2024 · similarity 0.90)
- ZeroSep: Separate Anything in Audio with Zero Training (NeurIPS 2025 · similarity 0.88)
- MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction (NeurIPS 2025 · similarity 0.88)
- Continual Audio-Visual Sound Separation (NeurIPS 2024 · similarity 0.88)
- Customized Condition Controllable Generation for Video Soundtrack (CVPR 2025 · similarity 0.87)
- MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
