Research map / Label-efficient and robust learning / Recognition and long-tail
Supervised Contrastive Learning from Weakly-Labeled Audio Segments for Musical Version Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CLEWS, a segment-level contrastive learning method for musical version matching that uses pairwise segment distance reductions (e.g., best-pair without replacement) and a modified decoupled alignment-and-uniformity loss, achieving state-of-the-art track-level and strong segment-level results.
Problem
Musical version matching is usually done at track level using full-length tracks, while applications need segment-level (e.g., 20 s) matching; existing losses are classification or triplet, and track-level weak labels make segment learning hard.
Approach
Reduce segment-level distance matrices to track-level distances via new reduction functions (best-pair without replacement, bpwr-r) that avoid consecutive and full-track matching, and train with a supervised, decoupled, Euclidean variant of the alignment-and-uniformity loss with a smoothing constant epsilon.
Key results
- Track-level: CLEWS reaches NAR 2.70 on DVI-Test (vs. 3.69 for DVINet+) and MAP 0.876 on SHS-Test, a new state of the art on both sets.
- Segment-level best-match on DVI-Test with 20 s queries: CLEWS NAR 2.86 and MAP 0.754, far ahead of ByteCover3 and CQTNet.
- Loss ablation on DVI-Valid: CLEWS loss gets MAP 0.804 vs. 0.676 for SupCon and 0.684 for SigLIP; classification cross-entropy gets MAP 0.205.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn from weakly labeled segments using CLEWS | musical version matching · DVI-Test | NAR: 3.69 $\pm$ 0.06 -> 2.70 $\pm$ 0.05 | Helped |
| learn from weakly labeled segments using CLEWS | musical version matching · DVI-Test | MAP: 0.643 $\pm$ 0.002 -> 0.774 $\pm$ 0.002 | Helped |
| learn from weakly labeled segments using CLEWS | musical version matching · SHS-Test | MAP: 0.863 -> 0.876 $\pm$ 0.005 | Helped |
| use bpwr-3 rather than bpwr-5 for positive reductions (ablation) | CLEWS · DVI-Valid | MAP: 0.804 $\pm$ 0.003 -> \textbf{0.809} $\pm$ \textbf{0.003} | Helped |
| use bpwr-8 rather than bpwr-5 for positive reductions (ablation) | CLEWS · DVI-Valid | NAR: 2.57 $\pm$ 0.09 -> \textbf{2.51} $\pm$ \textbf{0.09} | Helped |
| use min rather than bpwr-5 for positive reductions (ablation) | CLEWS · DVI-Valid | NAR: 2.57 $\pm$ 0.09 -> 2.79 $\pm$ 0.09 | Hurt |
| use meanmin rather than bpwr-5 for positive reductions (ablation) | CLEWS · DVI-Valid | MAP: 0.804 $\pm$ 0.003 -> 0.798 $\pm$ 0.003 | Hurt |
| use best-10 rather than bpwr-5 for positive reductions (ablation) | CLEWS · DVI-Valid | MAP: 0.804 $\pm$ 0.003 -> 0.795 $\pm$ 0.003 | Hurt |
| use best-10 rather than min for negative reductions (ablation) | CLEWS · DVI-Valid | NAR: 2.57 $\pm$ 0.09 -> 2.82 $\pm$ 0.10 | Hurt |
| use bpwr-5 rather than min for negative reductions (ablation) | CLEWS · DVI-Valid | MAP: 0.804 $\pm$ 0.003 -> 0.778 $\pm$ 0.003 | Hurt |
| use meanmin rather than min for negative reductions (ablation) | CLEWS · DVI-Valid | MAP: 0.804 $\pm$ 0.003 -> 0.488 $\pm$ 0.004 | Hurt |
| use the CLEWS loss instead of SupCon (ablation) | CLEWS with default reductions · DVI-Valid | MAP: 0.676 $\pm$ 0.004 -> 0.804 $\pm$ 0.003 | Helped |
Surprising findings
- Classification cross-entropy performs very poorly on DVI (MAP 0.205) because most version groups have only a few items per class.
- DVINet+ ranks much higher on DVI-Test than on SHS-Test, which the authors attribute to classification losses being ineffective with few items per class.
Limitations
- ByteCover baselines are the authors' approximations, since the original models are reported as non-reproducible.
- Training on SHS takes about 2 days and DVI about 9 days on two H100 GPUs, which is costly.
- Hyper-parameters were not extensively tuned; results depend on gamma and epsilon choices with trade-off between NAR and MAP.
Experimental setup
Datasets: DiscogsVI-YT SHS100k-v2
Benchmarks: DVI-Test SHS-Test
Metrics: NAR MAP
Baselines: CQTNet MOVE LyraC-Net CoverHunter DVINet+ ByteCover1/2 ByteCover3 ByteCover3.5 SupCon SigLIP Triplet
Builds on: alignment and uniformity loss SupCon ByteCover3 CoverHunter SigLIP
Closest papers by meaning
- Weakly-supervised Audio Separation via Bi-modal Semantic Similarity (ICLR 2024 · similarity 0.85)
- Contrastive Learning from Synthetic Audio Doppelgängers (ICLR 2025 · similarity 0.83)
- Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training (NeurIPS 2026 · similarity 0.83)
- Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation (ICLR 2024 · similarity 0.83)
- MCSSME: Multi-Task Contrastive Learning for Semi-supervised Singing Melody Extraction from Polyphonic Music (AAAI 2024 · similarity 0.82)
- MusicFlow: Cascaded Flow Matching for Text Guided Music Generation (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.