Research map / Label-efficient and robust learning / Recognition and long-tail
Personalized Dynamic Music Emotion Recognition with Dual-Scale Attention-Based Meta-Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DSAML, a dual-scale attention transformer with MAML and annotator-based task construction for personalized dynamic music emotion recognition (PDMER), predicting per-moment valence-arousal from one personalized annotated sample.
Problem
Dynamic music emotion recognition (DMER) struggles to capture long-term dependencies and ignores individual differences in emotion perception; the paper targets predicting personalized per-moment valence-arousal curves for new users with one annotated sample.
Approach
Combines a frozen ImageBind encoder with a trainable adapter, a dual-scale (local/global mask) attention transformer with a diagonal attention-map loss, and MAML where each task is built from one annotator's labels so that adaptation captures personal perception.
Key results
- On DEAM traditional DMER, DSAML reaches arousal CCC 0.39 and PCC 0.675 versus DAMFF's 0.354 and 0.621.
- On PDMER (DEAM), DSAML achieves arousal CCC 0.377 and valence CCC 0.092, versus DAMFF's 0.232 and 0.035.
- In a 22-participant subjective study, DSAML's arousal curves were ranked as most consistent with listener perception, even above the dataset ground truth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dual-scale attention instead of removing local attention (ablation) | DSAML · DEAM and PMEmo traditional DMER | CCC: 0.385 -> 0.39; 0.167 -> 0.186 | Helped |
| Use dual-scale attention instead of removing global attention (ablation) | DSAML · DEAM and PMEmo traditional DMER | CCC: 0.37 -> 0.39; 0.157 -> 0.186 | Helped |
| Add diagonal attention map loss (ablation) | DSAML · DEAM and PMEmo traditional DMER | CCC: 0.338 -> 0.39; 0.148 -> 0.186 | Helped |
| Add ImageBind Adapter (ablation) | DSAML · DEAM and PMEmo traditional DMER | CCC: 0.329 -> 0.39; 0.138 -> 0.186 | Helped |
| Use MAML (ablation) | DSAML with personalized task construction · DEAM PDMER, arousal | CCC: 0.305 -> 0.377 | Helped |
| Use annotator-based personalized task construction (ablation) | DSAML with MAML · DEAM PDMER, arousal | CCC: 0.357 -> 0.377 | Helped |
| Remove both MAML and personalized task construction (ablation) | DSAML · DEAM PDMER, arousal | CCC: 0.332 vs 0.377 | Hurt |
| Apply MAML to conventional DMER baselines | DAMFF, MCRNN, DNN and CRNN · DEAM PDMER | performance | Hurt |
| Use DSAML rather than DAMFF | Traditional DMER models · DEAM and PMEmo traditional DMER | CCC: 0.354 -> 0.39; 0.055 -> 0.104; 0.175 -> 0.186; 0.091 -> 0.108 | Helped |
| Use DSAML rather than DAMFF | PDMER models · DEAM PDMER | CCC: 0.22 -> 0.377; 0.035 -> 0.092 | Helped |
Surprising findings
- Applying MAML to baseline methods reduced their PDMER performance, since models fit the support-set personality but struggle with differing query-set perceptions.
- DSAML's predicted arousal curves were ranked higher than the dataset ground truth in the subjective study.
Limitations
- Only 1000 DEAM songs have annotator IDs, so personalized task construction cannot be applied to PMEmo.
- Valence prediction from audio alone remains weak (PDMER valence CCC 0.092).
Experimental setup
Datasets: DEAM PMEmo
Benchmarks: DEAM PMEmo
Metrics: CCC PCC RMSE
Baselines: CRNN DNN MCRNN DAMFF
Builds on: MAML ImageBind DAMFF BiLSTM
Closest papers by meaning
- MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion (AAAI 2024 · similarity 0.86)
- MCSSME: Multi-Task Contrastive Learning for Semi-supervised Singing Melody Extraction from Polyphonic Music (AAAI 2024 · similarity 0.83)
- OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition (ICML 2025 · similarity 0.83)
- Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning (NeurIPS 2024 · similarity 0.82)
- HarmonySet: A Comprehensive Dataset for Understanding Video-Music Semantic Alignment and Temporal Synchronization (CVPR 2025 · similarity 0.82)
- DMMR: Cross-Subject Domain Generalization for EEG-Based Emotion Recognition via Denoising Mixed Mutual Reconstruction (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.