Research map / Vision-language and multimodal / Cross-modal learning
USE: A Unified Model for Universal Sound Separation and Extraction
Hongyu Wang, Chenda Li, Xin Zhou, Shuai Wang, Yanmin Qian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on 2Mix and 3Mix with EDA (stage 1) instead of fixed 2Mix baseline (ablation) | Sepformer / BSRNN · AudioSet 2Mix Seen | SNRi: 7.4 -> 8.7 (Sepformer); 8.6 -> 9.2 (BSRNN) | Improved |
| add joint TSE training (stage 2) to separation training (ablation) | USE-S · AudioSet 3Mix Unseen | SNRi: 5.2 -> 6.3 | Improved |
| add joint TSE training (stage 2) to separation training (ablation) | USE-B · AudioSet 3Mix Unseen | SNRi: 4.8 -> 4.8 | No clear effect |
| add joint TSE training (stage 2) to separation training (ablation) | USE-B · AudioSet 2Mix Seen | SNRi: 9.2 -> 9.4 | Improved |
| joint SS+TSE training instead of TSE-only training (ablation) | USE-B · AudioSet 2Mix Seen TSE | SNRi: 8.4 -> 8.9 | Improved |
| joint SS+TSE training instead of TSE-only training (ablation) | USE-S · AudioSet 2Mix Seen TSE | SNRi: 8.5 -> 8.5 | No clear effect |
| joint SS+TSE training instead of TSE-only training (ablation) | USE-B · AudioSet 3Mix Unseen TSE | SNRi: 4.8 -> 5.0 | No clear effect |
| use all three clue modalities (text+tag+video) instead of a single clue (ablation) | USE-S · AudioSet 2Mix Seen TSE | SNRi: 6.8 -> 8.5 | Improved |
| predict source count with EDA instead of fixed known count (ablation) | USE-B · Multi-dataset separation | SNRi: AudioSet Seen 3Mix 6.7 -> 6.3; 2Mix 8.9 -> 9.2 | Mixed |
| use USE-B instead of TDCN++ | FUSS dry 4Mix · FUSS dry | SI-SNRi: 7.4 -> 11.9 | Improved |
| MSE plus InfoNCE alignment loss versus MSE only or InfoNCE only (ablation) | USE · AudioSet | alignment accuracy / SI-SNRi | Improved |
Closest papers by meaning
- OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup (ICLR 2025 · similarity 0.86)
- Weakly-supervised Audio Separation via Bi-modal Semantic Similarity (ICLR 2024 · similarity 0.85)
- Continual Audio-Visual Sound Separation (NeurIPS 2024 · similarity 0.84)
- SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes (ICLR 2025 · similarity 0.83)
- Independency Adversarial Learning for Cross-Modal Sound Separation (AAAI 2024 · similarity 0.83)
- Massive Sound Embedding Benchmark (MSEB) (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
