Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Multi-dataset Joint Pre-training of Emotional EEG Enables Generalizable Affective Computing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes mdJPT, a task-specific multi-dataset joint pre-training framework for EEG emotion recognition using a covariance-based cross-dataset alignment loss, contrastive inter-subject alignment, and a hybrid MLLA + spatiotemporal encoder.
Problem
Generic EEG foundation models underperform on emotion recognition due to task-feature mismatch and dilution; cross-dataset emotion EEG suffers from distribution shifts, inconsistent emotion label definitions, montage differences, and inter-subject variability, and existing methods need per-subject f
Approach
Jointly pre-train on multiple emotion EEG datasets without emotion labels, using a cross-dataset covariance centroid alignment loss (CDA) plus a temporally aligned contrastive inter-subject alignment loss (ISA), with an MLLA channel encoder followed by a spatiotemporal dynamics model.
Key results
- Few-shot leave-one-dataset-out: mdJPT averages 56.22% accuracy and 79.96 AUROC, reported as 4.57% AUROC above the best baseline (MMM 75.39).
- Zero-shot: mdJPT reaches 72.47% on SEED and 73.34% on DEAP, averaging about 11.9% accuracy improvement over the next-best method.
- Scaling with pre-training datasets on SEED-V target: multi-dataset joint pre-training gives an 8.55% gain over the best single-dataset setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mdJPT instead of the strongest listed baseline, MMM | Cross-dataset few-shot evaluation · Cross-dataset few-shot, average | accuracy: 54.54 -> 56.22 | Helped |
| Use mdJPT instead of the strongest listed baseline, MMM | Cross-dataset few-shot evaluation · Cross-dataset few-shot, average | AUROC: 75.39 -> 79.96 | Helped |
| Use mdJPT instead of the strongest listed baseline | Zero-shot evaluation · SEED, SEED-IV, SEED-V, SEED-VII, DEAP, and FACED | accuracy: 72.47 vs 55.42; 50.59 vs 46.86; 52.91 vs 43.32; 33.26 vs 23.58; 73.34 vs 67.04; 17.44 vs 11.64 | Helped |
| Use multiple pre-training datasets rather than the best single-dataset setting | mdJPT · SEED-V few-shot | accuracy: 8.55% | Helped |
| Add CDA loss with a small positive factor rather than factor 0 (ablation) | mdJPT · SEED-V few-shot | accuracy: 63.52 -> 64.98 | Helped |
| Set CDA loss factor to 0.020 (ablation) | mdJPT · SEED-V few-shot | accuracy: 63.52 -> 65.02 | Helped |
| Increase CDA loss factor beyond 0.020 (ablation) | mdJPT · SEED-V few-shot | accuracy: 65.02 -> 64.70; 64.10; 64.08 | Hurt |
| Add ISA loss (ablation) | mdJPT · SEED-V few-shot | accuracy: 30.51 ± 1.16 -> 62.35 ± 4.10 | Helped |
| Use MLLA instead of a vanilla Transformer channel encoder (ablation) | mdJPT · SEED-V few-shot | accuracy: 59.90 ± 1.32 -> 62.35 ± 4.10 | Helped |
| Use spatiotemporal dynamics model instead of a Transformer layer (ablation) | mdJPT · SEED-V few-shot | accuracy: 63.63 ± 0.96 -> 65.02 ± 0.98 | Helped |
| Use ISA loss instead of supervised contrastive learning (ablation) | mdJPT · SEED-V few-shot | accuracy: 47.99 ± 2.51 -> 65.02 ± 0.98 | Helped |
| Align positive samples temporally (ablation) | mdJPT · SEED-V few-shot | accuracy: 55.47 ± 1.71 -> 65.02 ± 0.98 | Helped |
Surprising findings
- Removing the ISA loss drops SEED-V accuracy to 30.51%, below the DE baseline.
- Supervised contrastive learning with emotion labels (47.99%) performs far worse than the label-free ISA loss (65.02%) on SEED-V.
Limitations
- Performance on the nine-class FACED task remains unsatisfactory, with residual distribution shift.
- Evaluation focuses on video-induced emotion; imagery-induced transfer on EmoEEG-MC is low (20.56% accuracy).
- Does not model individual differences in emotional experience and is limited by demographic diversity of datasets.
Experimental setup
Datasets: SEED SEED-IV SEED-V SEED-VII FACED DEAP EmoEEG-MC
Benchmarks: SEED SEED-IV SEED-V SEED-VII FACED DEAP EmoEEG-MC
Metrics: accuracy precision recall F1 score AUROC
Baselines: MMM LaBraM EEGPT DE baseline Transformer Supervised CL MK-MMD loss
Builds on: Mamba-like linear attention (MLLA) SupCon LaBraM EEGPT
Closest papers by meaning
- DMMR: Cross-Subject Domain Generalization for EEG-Based Emotion Recognition via Denoising Mixed Mutual Reconstruction (AAAI 2024 · similarity 0.90)
- FEEL: Quantifying Heterogeneity in Physiological Signals for Generalizable Emotion Recognition (NeurIPS 2025 · similarity 0.87)
- EEVR: A Dataset of Paired Physiological Signals and Textual Descriptions for Joint Emotion Representation Learning (NeurIPS 2024 · similarity 0.87)
- Multi-to-Single: Reducing Multimodal Dependency in Emotion Recognition Through Contrastive Learning (AAAI 2025 · similarity 0.86)
- Beyond Mimicking Under-Represented Emotions: Deep Data Augmentation with Emotional Subspace Constraints for EEG-Based Emotion Recognition (AAAI 2024 · similarity 0.85)
- REVE: A Foundation Model for EEG - Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.