Research map / Label-efficient and robust learning / Self-supervised and contrastive

Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation

NeurIPS 2026 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Masked autoencoder pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MTDP, a two-stage multi-teacher distillation framework (learned gating over frozen DINOv3 and Chronos teachers, then distillation into CBraMod) that pretrains EEG foundation models and outperforms masked-reconstruction pretraining on 12 EEG datasets.

Problem

Self-supervised masked reconstruction is the dominant EEG foundation model pretraining paradigm, but EEG data is scarce and low SNR, so reconstruction may focus on noise and scale poorly.

Approach

Use frozen cross-modal foundation models (vision DINOv3 and time-series Chronos) as teachers; fuse their representations with a learnable gating network trained by a masked latent denoising objective, then distill the fused target into an EEG student with a cosine loss.

Key results

• CBraMod-MTDP (100% TUEG) improves over CBraMod on BCIC-IV-2a by +8.43 balanced accuracy and on BCIC2020-3 by +8.80 balanced accuracy.
• CBraMod-MTDP 25% outperforms CBraMod 100%-reconstruction-pretrained on 9 of 12 datasets' comparisons described (e.g., CHB-MIT B.Acc 80.13 vs 73.98), using only 25% of pretraining data.
• In linear probing, DINOv3 outperforms CBraMod on all datasets except CHB-MIT; in ablation, the fused two-teacher MTDP gives the best results on FACED, PhysioNet-MI, SHU-MI and BCIC2020-3 (e.g., BCIC2020-3 59.87 vs 37.73 for Chronos-only).

Surprising findings

• A vision foundation model (DINOv3) applied to min-max normalized EEG as single-channel images outperforms the EEG-specific CBraMod under linear probing on most datasets.
• Naively summing distillation losses from both teachers does not consistently beat single-teacher distillation, while learned gating does.

Limitations

• Teacher inference and storage of precomputed representations add computational, disk and I/O cost.
• Preprocessing of EEG for teachers is minimal (min-max normalization and channel replication); alternative representations like spectrograms or wavelets gave no significant gain.
• Results are reported for a single student architecture (CBraMod) and some tasks (SHU-MI, ISRUC) show no gain or slight drops at 100% pretraining data.

Experimental setup

Datasets: TUEG PhysioNet-MI SHU-MI BCIC-IV-2a FACED SEED-V ISRUC CHB-MIT BCIC2020-3 Mumtaz2016 MentalArithmetic TUEV

Benchmarks: FACED PhysioNet-MI SHU-MI BCIC2020-3 Mumtaz2016 MentalArithmetic ISRUC TUEV CHB-MIT TUAB SEED-V BCIC-IV-2a

Metrics: balanced accuracy AUC-PR AUROC Cohen's kappa weighted F1

Baselines: CBraMod LaBraM BIOT EEGNet EEGConformer SPaRCNet ContraWR CNN-Transformer FFCL ST-Transformer

Builds on: CBraMod AM-RADIO Chronos DINOv3

Closest papers by meaning

• What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.91)
• B[FM]$^2$: Brain Foundation Model via Flow Matching with SplitUNet (NeurIPS 2026 · similarity 0.89)
• NeurIPT: Foundation Model for Neural Interfaces (NeurIPS 2025 · similarity 0.89)
• REVE: A Foundation Model for EEG - Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects (NeurIPS 2025 · similarity 0.89)
• BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.88)
• Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG (NeurIPS 2026 · similarity 0.88)