Research map / Large language models / Language, safety and interpretability
Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaBraM, a large EEG foundation model pre-trained on ~2,500 hours of multi-dataset EEG via a VQ neural-spectrum tokenizer and masked neural-token prediction with a neural Transformer.
Problem
EEG deep learning models are task- and dataset-specific and small, because EEG datasets are small and heterogeneous in channel count, length, and task design, limiting generalizability across BCI tasks.
Approach
Segment multi-channel EEG into channel patches with learnable temporal and spatial embeddings so any electrode configuration fits one Transformer; train a vector-quantized tokenizer by predicting Fourier amplitude and phase, then pre-train by masked prediction of the discrete neural codes, with symmetric masking to reuse tokenizer forward passes.
Key results
- On TUAB, LaBraM-Huge reaches 0.8258 balanced accuracy and 0.9162 AUROC, above the best baseline BIOT (0.7959 / 0.8815).
- On TUEV, LaBraM-Huge reaches 0.6616 balanced accuracy and 0.6745 Cohen's kappa, versus 0.5281 and 0.5273 for BIOT.
- On SEED-V, LaBraM-Huge reaches 0.4102 accuracy versus 0.3802 for BIOT; on MoBI, LaBraM-Huge reaches 0.5632 Pearson correlation versus 0.5442 for ST-Transformer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale LaBraM from Base to Huge | LaBraM · TUAB | AUROC: 0.9022$\pm$0.0009 -> 0.9162$\pm$0.0016 | Helped |
| Scale LaBraM from Base to Huge | LaBraM · TUEV | Weighted F1: 0.8312$\pm$0.0052 -> 0.8329$\pm$0.0086 | No clear effect |
| Scale LaBraM from Base to Huge | LaBraM · SEED-V | Accuracy: 0.4095$\pm$0.0062 -> 0.4102$\pm$0.0037 | No clear effect |
| Scale LaBraM from Base to Huge | LaBraM · MoBI | R2 Score: 0.2876$\pm$0.0032 -> 0.3145$\pm$0.0032 | Helped |
| Use vector-quantized neural spectrum prediction rather than directly predicting tokenizer embeddings (ablation) | LaBraM masked EEG modeling · TUEV | Weighted F1: 0.8170$\pm$0.0058 -> 0.8312$\pm$0.0052 | Helped |
| Use vector-quantized neural spectrum prediction rather than directly predicting tokenizer embeddings (ablation) | LaBraM masked EEG modeling · TUAB | AUROC: 0.8964$\pm$0.0012 -> 0.9022$\pm$0.0009 | No clear effect |
| Use the neural tokenizer and codebook instead of reconstructing raw EEG patches with MSE (ablation) | LaBraM masked EEG modeling · TUEV | Weighted F1: 0.7979$\pm$0.0082 -> 0.8312$\pm$0.0052 | Helped |
| Use the neural tokenizer and codebook instead of reconstructing Fourier spectrum with MSE (ablation) | LaBraM masked EEG modeling · TUEV | Weighted F1: 0.7819$\pm$0.0040 -> 0.8312$\pm$0.0052 | Helped |
| Use symmetric mask ratios 0.5 and 0.5 instead of 0.4 and 0.6 (ablation) | LaBraM · TUEV | Weighted F1: 0.8067$\pm$0.0059 -> 0.8312$\pm$0.0052 | Helped |
| Use symmetric mask ratios 0.5 and 0.5 instead of 0.4 and 0.6 (ablation) | LaBraM · TUAB | AUROC: 0.9049$\pm$0.0038 -> 0.9022$\pm$0.0009 | No clear effect |
| Use symmetric masking rather than single masking (ablation) | LaBraM · TUAB and TUEV | Downstream performance | Mixed |
| Include downstream task datasets in pre-training (ablation) | LaBraM · TUAB and TUEV | Downstream performance | No clear effect |
Surprising findings
- Pre-training with the downstream datasets included made little difference to downstream performance.
- Naive masked reconstruction of raw EEG or its Fourier spectrum performed slightly better on TUAB, while the tokenizer-based variant won on the higher-level TUEV task.
Limitations
- Pre-training data is only ~2,500 hours, which the authors say is small relative to NLP and vision corpora.
- Phase reconstruction is noticeably worse than amplitude reconstruction in the tokenizer.
Experimental setup
Datasets: TUAB TUEV BCI Competition IV-1 Emobrain Grasp and Lift EEG Challenge Inria BCI Challenge EEG Motor Movement/Imagery Dataset SEED series TUEG subsets (TUAR, TUEP, TUSZ, TUSL) Siena Scalp EEG Database self-collected EEG
Benchmarks: TUAB TUEV SEED-V MoBI
Metrics: balanced accuracy AUC-PR AUROC Cohen's kappa weighted F1 accuracy Pearson's correlation R2 score RMSE
Baselines: SPaRCNet ContraWR CNN-Transformer FFCL ST-Transformer BIOT
Builds on: VQ-VAE BEiT masked autoencoders
Closest papers by meaning
- CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding (ICLR 2025 · similarity 0.88)
- EEG-Language Pretraining for Highly Label-Efficient Clinical Phenotyping (ICML 2025 · similarity 0.87)
- NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals (ICLR 2025 · similarity 0.86)
- What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.86)
- NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces (NeurIPS 2026 · similarity 0.86)
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.