Research map / Label-efficient and robust learning / Semi-supervised learning
MCSSME: Multi-Task Contrastive Learning for Semi-supervised Singing Melody Extraction from Polyphonic Music
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MCSSME, a multi-task contrastive semi-supervised framework for singing melody extraction that combines self-consistency regularization across pyramid layers, transformation recognition, contrastive embedding learning, and domain-adversarial classification, built on MSNet.
Problem
Singing melody extraction from polyphonic music is limited by scarce pixel-level labels and by performance degradation across music genres.
Approach
Uses multi-level predictions of a pyramid-shaped feature extractor as self-consistency targets to select pseudo-labels on unlabeled audio, jointly with signal-transformation classification, contrastive embedding learning (SupCon-style for labeled data), and a domain classifier separating popular and classical genres.
Key results
- MCSSME reaches 76.7% OA on ADC2004 versus 72.4% for FTANet and 73.1% for Teacher-student.
- MCSSME reaches 85.6% OA on MIREX 05 and 84.7% OA on iKala, outperforming FTANet (84.4% and 80.3%).
- On MedleyDB, MCSSME achieves 73.4% OA versus 68.8% for FTANet; ablations show removing SCR drops ADC2004 OA by 3.5 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MCSSME instead of FTANet | Singing melody extraction · ADC2004 | OA: 72.4 -> 76.7 | Helped |
| use MCSSME instead of FTANet | Singing melody extraction · MIREX 05 | OA: 84.4 -> 85.6 | Helped |
| use MCSSME instead of FTANet | Singing melody extraction · MedleyDB | OA: 68.8 -> 73.4 | Helped |
| use MCSSME instead of FTANet | Singing melody extraction · iKala | OA: 80.3 -> 84.7 | Helped |
| add domain classifier (ablation) | MCSSME · ADC2004 | OA: 73.1 -> 76.7 | Helped |
| add self-consistency regularization (ablation) | MCSSME · ADC2004 | OA: 74.0 -> 76.7 | Helped |
| add contrastive learning (ablation) | MCSSME · ADC2004 | OA: 74.7 -> 76.7 | Helped |
| add signal transformation (ablation) | MCSSME · ADC2004 | OA: 75.3 -> 76.7 | Helped |
| add domain classifier (ablation) | MCSSME · MIREX 05 | OA: 85.3 -> 85.6 | Helped |
| add self-consistency regularization (ablation) | MCSSME · MIREX 05 | OA: 81.9 -> 85.6 | Helped |
| add contrastive learning (ablation) | MCSSME · MIREX 05 | OA: 83.2 -> 85.6 | Helped |
| add signal transformation (ablation) | MCSSME · MIREX 05 | OA: 84.0 -> 85.6 | Helped |
Surprising findings
- The model produces more melody detection errors (non-melody frames predicted as melody) than MSNet, which the authors attribute to the signal transformations.
Limitations
- Higher rate of voicing false alarms / melody detection errors than MSNet on opera examples.
- Gains on the MIREX 05 domain are smaller, and the domain classifier helps less on popular music.
- Relies on a limited set of hand-designed signal transformations (noise, filtering, rotation, shuffle).
Experimental setup
Datasets: MIR-1K MedleyDB FMA RWC
Benchmarks: ADC2004 MIREX 05 MedleyDB iKala
Metrics: overall accuracy raw pitch accuracy raw chroma accuracy voicing recall voicing false alarm
Baselines: DSM MSNet MD+MR Teacher-student FTANet HGNet
Builds on: MSNet SimCLR SupCon Mean Teacher Teacher-student semi-supervised melody extraction
Closest papers by meaning
- GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks (NeurIPS 2024 · similarity 0.83)
- MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training (ICLR 2024 · similarity 0.83)
- JEN-1 DreamStyler: Customized Musical Concept Learning via Pivotal Parameters Tuning (AAAI 2025 · similarity 0.83)
- Personalized Dynamic Music Emotion Recognition with Dual-Scale Attention-Based Meta-Learning (AAAI 2025 · similarity 0.83)
- LLark: A Multimodal Instruction-Following Language Model for Music (ICML 2024 · similarity 0.83)
- SongGLM: Lyric-to-Melody Generation with 2D Alignment Encoding and Multi-Task Pre-Training (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.