Research map / Architectures and efficiency / Attention and transformers
CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CSBrain, an EEG foundation model that alternates cross-scale spatiotemporal tokenization with structured sparse attention, pretrained with masked autoencoding on TUEG and evaluated on 11 BCI tasks across 16 datasets.
Problem
Existing EEG foundation models use scale-agnostic fixed-size tokens and dense attention, which mismatches the cross-scale spatiotemporal structure of EEG and introduces spurious dependencies and computational overhead.
Approach
Multi-scale temporal and spatial convolution tokenization within time windows and brain regions (CST), alternated with structured sparse attention that attends across temporal windows (grouped by relative position) and across brain regions (regional descriptors) instead of densely across all tokens (SSA).
Key results
- CSBrain reaches a macro-average of 0.7095 across tasks, versus 0.6760 for CBraMod, 0.6697 for LaBraM and 0.6322 for BIOT.
- On BCIC-IV-2a balanced accuracy, CSBrain scores 0.5657 versus 0.5138 for CBraMod.
- On the TUSL dataset, CSBrain reaches 0.8571 balanced accuracy versus 0.7625 for the best prior baseline (LaBraM).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cross-scale tokenization with K=3 instead of single-scale tokenization with K=1 (ablation) | CSBrain CST · Four representative tasks: emotion recognition, motor imagery, event classification, imagined speech | performance: 10.8%, 20.7%, and 12.1% | Helped |
| Stack CST before each SSA module instead of applying CST only once before the first SSA (ablation) | CSBrain · Four representative tasks: emotion recognition, motor imagery, event classification, imagined speech | performance | Helped |
| Use structured sparse attention instead of criss-cross attention (ablation) | CSBrain · TUEV | B-Acc and Weighted F1: +8.1% B-Acc and +5.3% Weighted F1 | Helped |
| Use structured sparse attention instead of dense or criss-cross attention (ablation) | CSBrain · Four attention-comparison tasks | Balanced Accuracy, Cohen's Kappa, Weighted F1 | Helped |
| Use CSBrain instead of CBraMod | EEG decoding across the evaluated datasets · Macro-average | score: 0.6760 -> 0.7095 | Helped |
Surprising findings
- Task-specific models trained from scratch perform far below foundation models on nearly all datasets reported.
Limitations
- Some baselines were re-implemented and retrained under a unified protocol, with minor per-dataset learning-rate adjustments.
- Spatial regions are proxied by 10-20 electrode labels rather than true anatomical regions.
Experimental setup
Datasets: TUEG BCIC-IV-2a PhysioNet-MI SHU-MI FACED SEED-V CHB-MIT Siena ISRUC HMC BCIC2020-3 SEED-VIG
Benchmarks: BCIC-IV-2a SHU-MI PhysioNet-MI BCIC2020-3 SEED-V FACED TUEV TUAB MentalArithmetic CHB-MIT Siena SEED-VIG
Metrics: balanced accuracy weighted F1 AUROC AUC-PR Cohen's kappa Pearson correlation R2
Baselines: EEGNet EEGConformer SPaRCNet ContraWR CNN-Transformer FFCL ST-Transformer BIOT LaBraM CBraMod
Builds on: masked autoencoding LaBraM CBraMod BIOT
Closest papers by meaning
- CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding (ICLR 2025 · similarity 0.90)
- BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.89)
- NeurIPT: Foundation Model for Neural Interfaces (NeurIPS 2025 · similarity 0.89)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.89)
- A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks (NeurIPS 2025 · similarity 0.89)
- Neural decoding from stereotactic EEG: accounting for electrode variability across subjects (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.