Research map / Architectures and efficiency / Attention and transformers
CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CBraMod, an EEG foundation model with a criss-cross transformer (parallel spatial and temporal attention) and asymmetric conditional positional encoding, pre-trained on TUEG via masked patch reconstruction and evaluated on 10 downstream BCI tasks across 12 datasets.
Problem
Existing EEG foundation models model all EEG patches jointly (full attention), ignoring heterogeneous spatial and temporal dependencies, and use absolute channel-based positional encodings that generalize poorly across varying EEG formats and montages.
Approach
A criss-cross transformer that splits attention heads into parallel spatial (within time interval) and temporal (within channel) attention, combined with an asymmetric depthwise 2D convolutional conditional positional encoding (kernel larger in the channel dimension) that adapts to arbitrary EEG channel counts and lengths.
Key results
- On FACED 9-class emotion recognition, CBraMod reaches Cohen's Kappa 0.5041 vs. 0.4698 for LaBraM-Base.
- On PhysioNet-MI 4-class motor imagery, CBraMod reaches Cohen's Kappa 0.5222 vs. 0.4912 for LaBraM-Base.
- On CHB-MIT seizure detection, CBraMod reaches AUROC 0.8892 vs. 0.8761 for BIOT (best baseline).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use criss-cross attention instead of full attention (ablation) | CBraMod · emotion recognition and motor imagery datasets | performance | Helped |
| use criss-cross attention instead of axial attention (ablation) | CBraMod · emotion recognition and motor imagery datasets | performance | Helped |
| use dual parallel spatial and temporal attention instead of CCNet attention (ablation) | CBraMod · emotion recognition and motor imagery datasets | performance | Helped |
| add positional encoding rather than use no positional encoding (ablation) | CBraMod · emotion recognition and motor imagery datasets | performance | Helped |
| replace no positional encoding with absolute positional encoding (ablation) | CBraMod · emotion recognition and motor imagery datasets | performance | Helped |
| replace ACPE with CPE (ablation) | CBraMod · emotion recognition and motor imagery datasets | performance | Hurt |
| drop bad samples from TUEG during pre-training instead of retaining them (ablation) | CBraMod · FACED, SEED-V, PhysioNet-MI, and SHU-MI | performance: 0.4768 -> 0.5041; 0.2224 -> 0.2569; 0.5063 -> 0.5222; 0.6895 -> 0.6988 | Helped |
| add TUEG pre-training instead of training only on downstream data (ablation) | CBraMod · FACED, SEED-V, PhysioNet-MI, and SHU-MI | performance: 0.4615 -> 0.5041; 0.2211 -> 0.2569; 0.4994 -> 0.5222; 0.6878 -> 0.6988 | Helped |
| pre-train on dirty TUEG rather than skip pre-training (ablation) | CBraMod · FACED, SEED-V, PhysioNet-MI, and SHU-MI | performance: 0.4615 -> 0.4768; 0.2211 -> 0.2224; 0.4994 -> 0.5063; 0.6878 -> 0.6895 | Helped |
Surprising findings
- Full attention, the modeling strategy used by most EEG foundation models, performed worst among the attention variants compared.
- Dirty (uncleaned) pre-training performed only slightly better than no pre-training, suggesting noisy data weakens pre-training.
Limitations
- Pre-training uses a fixed set of 19 common channels, so the model cannot directly handle channel configurations outside this set during pre-training.
- BIOT baseline requires splitting >18-channel inputs into multiple models, which affects baseline comparisons.
- Pre-training data is clinical TUEG recordings, which may not match BCI-oriented recording conditions.
Experimental setup
Datasets: TUEG FACED SEED-V PhysioNet-MI SHU-MI ISRUC CHB-MIT BCIC2020-3 Mumtaz2016 SEED-VIG MentalArithmetic TUEV
Benchmarks: FACED SEED-V PhysioNet-MI SHU-MI ISRUC CHB-MIT BCIC2020-3 Mumtaz2016
Metrics: balanced accuracy cohen's kappa weighted F1 AUC-PR AUROC pearson correlation R2 RMSE
Baselines: EEGNet EEGConformer SPaRCNet ContraWR CNN-Transformer FFCL ST-Transformer BIOT LaBraM-Base DeepSleepNet USleep
Builds on: masked autoencoder conditional positional encoding criss-cross attention (CCNet) LaBraM BIOT
Closest papers by meaning
- What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.90)
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.90)
- BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.90)
- NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces (NeurIPS 2026 · similarity 0.89)
- NeurIPT: Foundation Model for Neural Interfaces (NeurIPS 2025 · similarity 0.89)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.