Research map / Label-efficient and robust learning / Self-supervised and contrastive
BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BrainOmni, a brain foundation model that jointly pretrains on EEG and MEG via a sensor-aware discrete tokenizer (BrainTokenizer) and masked token prediction with a Criss-Cross Transformer.
Problem
Existing brain models are modality- and device-specific; EEG and MEG have distinct signal patterns and heterogeneous sensor layouts, limiting cross-device, cross-modality generalisation, and MEG foundation models are largely unexplored.
Approach
A Sensor Encoder embeds sensor position, orientation and type so that a cross-attention channel compression maps arbitrary EEG/MEG channels into a fixed set of latent source variables, which are RVQ-quantised by a SEANet-based BrainTokenizer; a Criss-Cross Transformer then performs masked token prediction over these tokens for joint EMEG pretraining.
Key results
- BrainOmni_base reaches 0.828 BACC on AD65 and 0.748 on PD31, roughly 10% above the best pretrained baseline LaBraM on those datasets.
- On the MEG ASD74 dataset BrainOmni_tiny and base reach 0.621 and 0.651 BACC, beating the strongest baseline SPaRCNet at 0.604.
- Joint EMEG pretraining improves over MEG-only pretraining by about 12% relative BACC on ASD74 (0.621 vs 0.553).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly pretrain on EEG and MEG instead of EEG only (ablation) | BrainOmni_tiny · TUAB | BACC: 0.811 $\pm$ 0.006 -> 0.819 $\pm$ 0.004 | Helped |
| jointly pretrain on EEG and MEG instead of EEG only (ablation) | BrainOmni_tiny · TUEV | BACC: 0.583 $\pm$ 0.020 -> 0.603 $\pm$ 0.024 | Helped |
| jointly pretrain on EEG and MEG instead of MEG only (ablation) | BrainOmni_tiny · ASD74 | BACC: 0.553 $\pm$ 0.062 -> 0.621 $\pm$ 0.048 | Helped |
| use combined EEG and MEG input instead of EEG-only input (ablation) | EMEG-pretrained BrainOmni_tiny · SomatoMotor | BACC: 0.783 $\pm$ 0.076 -> 0.863 $\pm$ 0.128 | Helped |
| remove sensor embedding (ablation) | BrainOmni_tiny · TUAB, AD65, ASD74, SomatoMotor | BACC: TUAB 0.819 $\pm$ 0.004 -> 0.784 $\pm$ 0.005; AD65 0.795 $\pm$ 0.030 -> 0.730 $\pm$ 0.064; ASD74 0.621 $\pm$ 0.048 -> 0.5 | Hurt |
| remove sensor embedding (ablation) | BrainOmni_tiny · TUEV | BACC: 0.603 $\pm$ 0.024 -> 0.607 $\pm$ 0.016 | No clear effect |
| treat channels as uncorrelated single-channel data (ablation) | BrainOmni_tiny · TUAB, TUEV, AD65, ASD74, SomatoMotor | BACC: TUAB 0.819 $\pm$ 0.004 -> 0.788 $\pm$ 0.003; TUEV 0.603 $\pm$ 0.024 -> 0.526 $\pm$ 0.016; AD65 0.795 $\pm$ 0.030 -> 0.76 | Hurt |
| replace latent source estimation with electrode-level self-attention (ablation) | BrainOmni_tiny · TUAB, PD31, AD65, ASD74, SomatoMotor | BACC: TUAB 0.819 $\pm$ 0.004 -> 0.813 $\pm$ 0.003; PD31 0.736 $\pm$ 0.116 -> 0.627 $\pm$ 0.158; AD65 0.795 $\pm$ 0.030 -> 0.76 | Hurt |
| add frequency-domain loss (ablation) | BrainTokenizer with time-domain and commitment losses · BrainTokenizer reconstruction | MAE: 0.799 -> 0.471 | Helped |
| add PCC loss alongside frequency-domain loss (ablation) | BrainTokenizer with frequency-domain, time-domain, and commitment losses · BrainTokenizer reconstruction | PHASE: 1.583 -> 1.495 | Helped |
| add frequency-domain loss alongside PCC loss (ablation) | BrainTokenizer with PCC, time-domain, and commitment losses · BrainTokenizer reconstruction | PHASE: 1.863 -> 1.495 | Helped |
| use EEG-MEG joint pretraining instead of EEG-only pretraining (ablation) | BrainOmni_tiny · SomatoMotor EEG input | BACC: 0.766 $\pm$ 0.070 -> 0.783 $\pm$ 0.076 | Helped |
Surprising findings
- Removing the Sensor Encoder barely changes TUEV performance (0.607 without vs 0.603 with), while hurting MEG and EMEG tasks.
- With frozen backbone, BrainOmni_tiny on TUAB (0.800) is close to full fine-tuning of several baselines.
Limitations
- Pretraining data scale is limited, with only 656 hours of MEG.
- MEG and EMEG downstream evaluation is constrained by few datasets and lack of standardised test paradigms.
- The model does not always outperform task-specific models on individual tasks, e.g. PhysioNet-MI where CBraMod is best.
Experimental setup
Datasets: MEG-MASC MEG-Narrative-Dataset OMEGA CC700 Go-Nogo MusicEEG HFO AD65 MDD PD31 TUAB TUEV
Benchmarks: AD65 MDD PD31 TUAB TUEV FACED WBCIC_SHU PhysioNet-MI ASD74 MEG-MMI SomatoMotor
Metrics: balanced accuracy MSE MAE PCC
Baselines: CNN-Transformer ContraWR SPaRCNet ST-Transformer FAMED LaBraM CBraMod
Builds on: LaBraM CBraMod SEANet residual vector quantisation RoPE
Closest papers by meaning
- Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG (NeurIPS 2026 · similarity 0.91)
- NeurIPT: Foundation Model for Neural Interfaces (NeurIPS 2025 · similarity 0.90)
- CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding (ICLR 2025 · similarity 0.90)
- NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces (NeurIPS 2026 · similarity 0.90)
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.89)
- What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.