Research map / Label-efficient and robust learning / Self-supervised and contrastive
EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG Signals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EEGPT, a ~10M-parameter (25M in the reported large variant) transformer pretrained with dual self-supervised learning (masked reconstruction plus spatio-temporal representation alignment) on mixed EEG datasets, evaluated with linear probing on downstream EEG tasks.
Problem
Learning robust, universal EEG representations is hard due to low SNR, high inter-subject variability, task variation, and channel/sampling-rate mismatch across devices.
Approach
Combines BERT-style masked patch reconstruction with an explicit spatio-temporal alignment loss, where a momentum encoder provides targets from full (unmasked) EEG tokens, plus a local spatio-temporal patch embedding with learnable channel embeddings and a hierarchical spatial-then-temporal encoder/predictor design.
Key results
- On TUEV, EEGPT (25M) reaches 0.6232 balanced accuracy vs 0.5281 for BIOT, a 9.5-point gain.
- On TUAB, EEGPT reaches 0.7983 balanced accuracy, roughly matching BIOT (0.7959) and below LaBraM (0.8140) in the appendix.
- Across BCIC-2A, BCIC-2B, Sleep-EDFx, KaggleERN and PhysioP300, EEGPT with linear probing reports the best balanced accuracy or AUROC/F1 in most comparisons against BENDR, BIOT and LaBraM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EEGPT instead of BIOT | EEG classification · TUEV | Balanced Accuracy: 0.5281±0.0225 -> 0.6232±0.0114 | Helped |
| Use EEGPT instead of BIOT | EEG classification · TUEV | Weighted F1: 0.7492±0.0082 -> 0.8187±0.0063 | Helped |
| Use EEGPT instead of LaBraM | motor imagery classification · BCIC-2A | Balanced Accuracy: 0.5613±0.0052 -> 0.5846±0.0070 | Helped |
| Use EEGPT instead of BENDR | motor imagery classification · BCIC-2B | Balanced Accuracy: 0.7067±0.0011 -> 0.7212±0.0019 | Helped |
| Remove alignment loss (ablation) | EEGPT pretraining · BCIC-2A | Balanced Accuracy: 0.5846±0.0070 -> 0.5287±0.0086 | Hurt |
| Remove alignment loss (ablation) | EEGPT pretraining · BCIC-2B | AUROC: 0.8059±0.0032 -> 0.7264±0.0381 | Hurt |
| Remove layer normalization on reconstruction targets (ablation) | EEGPT pretraining · KaggleERN | AUROC: 0.6621±0.0096 -> 0.5891±0.0227 | Hurt |
| Remove the encoder-to-reconstructor skip connection (ablation) | EEGPT pretraining · BCIC-2A | Balanced Accuracy: 0.5846±0.0070 -> 0.5796±0.0011 | Hurt |
| Remove the encoder-to-reconstructor skip connection (ablation) | EEGPT pretraining · BCIC-2B | AUROC: 0.8059±0.0032 -> 0.7702±0.0122 | Hurt |
| Remove the predictor (ablation) | EEGPT pretraining · Pretraining validation | reconstruction loss | Hurt |
| Use linear probing instead of full fine-tuning (ablation) | EEGPT downstream evaluation · BCIC-2B | AUROC: 0.7871±0.0054 -> 0.7974±0.0030 | Helped |
| Add an adaptive spatial filter (ablation) | EEGPT downstream evaluation · KaggleERN | AUROC: 0.6078±0.0101 -> 0.6110±0.0019; 0.6463±0.0081 -> 0.6621±0.0096 | Helped |
Surprising findings
- On TUAB, the 25M EEGPT model is only comparable to BIOT and is below LaBraM (0.8140 BAC) in the appendix comparison.
- On BCIC-2A, the 4-class cross-subject result required combining the model with a task-specific MSConvNet branch, and the authors note the pure model is below task-specific SOTA.
Limitations
- Performance on BCIC-2A requires combining with a task-specific model; pure EEGPT lags task-specific SOTA.
- Comparable rather than superior on TUAB versus BIOT and LaBraM.
- Scaling-law claims rest on only eight variants and small parameter ranges.
Experimental setup
Datasets: PhysioMI HGD TSU SEED M3CV BCIC-2A BCIC-2B Sleep-EDFx KaggleERN PhysioP300 TUAB TUEV
Benchmarks: TUAB TUEV BCIC-2A BCIC-2B Sleep-EDFx KaggleERN PhysioP300
Metrics: balanced accuracy AUROC weighted F1 Cohen's Kappa
Baselines: BENDR BIOT LaBraM SPaRCNet ContraWR CNN-T FFCL ST-T
Builds on: masked autoencoders BERT Vision Transformer BENDR BIOT
Closest papers by meaning
- BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.89)
- Decoding Natural Images from EEG for Object Recognition (ICLR 2024 · similarity 0.88)
- CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding (ICLR 2025 · similarity 0.88)
- What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.87)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.87)
- Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.