Research map / Label-efficient and robust learning / Self-supervised and contrastive
The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pre-trains a SEANet-style MEG encoder with subject-conditioned FiLM on ~400 hours of unlabelled and labelled MEG using neuroscience-informed pretext tasks (band-stop prediction, phase shift, amplitude scaling), then trains a frozen-backbone linear probe for speech detection and voicing classification.
Problem
Speech decoding from non-invasive MEG is data-limited because single-subject datasets are small and heterogeneous datasets (different subjects, tasks, scanners) are hard to pool; the field has not exploited open neural data repositories at scale.
Approach
Self-supervised pretext tasks tailored to MEG (predicting a rejected frequency band, a random sensor-subset phase shift, and a random sensor-subset amplitude scale) that are agnostic to sensor count, combined with a dataset-conditional input layer and subject FiLM conditioning in a wave-to-wave convolutional encoder, allow pre-training on heterogeneous unlabelled MEG and frozen-representation line
Key results
- Best model reaches ROC AUC .705 for speech detection, a 15-27% gain over BIOT (.615), EEGPT (.602) and BrainBERT (.556) under linear probing.
- Matches surgical intracranial BrainBERT result of .71 using non-invasive MEG.
- Aggregating CamCAN and MOUS pre-training (319 h) improves ROC AUC to .638 over .630 (CamCAN alone).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-supervised pretext-task pre-training (ablation) | Ours · Heard speech MEG | ROC AUC: $.519_{\pm .002}$ -> $.630_{\pm .000}$ | Helped |
| combine band, phase, and amplitude pretext tasks instead of using one task alone (ablation) | Ours · Heard speech MEG | ROC AUC: $.624_{\pm .001}$ -> $.630_{\pm .000}$; $.615_{\pm .001}$ -> $.630_{\pm .000}$; $.588_{\pm .001}$ -> $.630_{\pm .000}$ | Helped |
| add Gaussian filtering to predictions (ablation) | Ours with all pretext tasks · Heard speech MEG | ROC AUC: $.630_{\pm .000}$ -> $\mathbf{.700}_{\pm .002}$ | Helped |
| use neuroscience-informed pretext objectives rather than generic SSL objectives | Speech detection model · Heard speech MEG | ROC AUC: $.615_{\pm .002}$ -> $\mathbf{.705}_{\pm .003}$ (BIOT); $.556_{\pm .007}$ -> $\mathbf{.705}_{\pm .003}$ (BrainBERT); $.6 | Helped |
| combine Cam-CAN and MOUS unlabelled pre-training data | Ours · Heard speech MEG | ROC AUC: $.630_{\pm .0001}$ -> $\mathbf{.638}_{\pm .0002}$; $p < 0.05$ | Helped |
| increase the volume of unlabelled Cam-CAN data | Ours · Speech decoding tasks | ROC AUC | Helped |
| increase the number of unlabelled-data subjects used for pre-training | Ours · Novel subjects | ROC AUC | Helped |
Surprising findings
- Pre-training on Cam-CAN, a resting and sensorimotor dataset with no speech task, outperformed pre-training on MOUS, which uses speech tasks on the same scanner as the downstream data.
- Performance keeps improving log-linearly with pre-training subjects and generalises to novel held-out subjects.
Limitations
- Evaluated only on speech detection and voicing classification, not full brain-to-text transcription.
- Only heard speech studied; attempted or imagined speech not tested.
- Only two MEG datasets aggregated; benefit of more datasets remains untested; spatial sensor geometry not exploited.
Experimental setup
Datasets: Cam-CAN MOUS Gwilliams et al. 2023 heard speech MEG (3 subjects, 30 h) Heard speech MEG dataset with 27 subjects (54 h)
Benchmarks: speech detection on MEG heard-speech datasets voicing classification
Metrics: ROC AUC
Baselines: BrainBERT BIOT EEGPT Linear probe on raw MEG No pre-training (random frozen encoder)
Builds on: SEANet FiLM BrainBERT BIOT
Closest papers by meaning
- LibriBrain100: One Hundred Hours of Broad and Deep MEG Data for Neural Speech Decoding at Scale (NeurIPS 2026 · similarity 0.91)
- Time-Masked Transformers with Lightweight Test-Time Adaptation for Neural Speech Decoding (NeurIPS 2025 · similarity 0.89)
- Towards Homogeneous Lexical Tone Decoding from Heterogeneous Intracranial Recordings (ICLR 2025 · similarity 0.87)
- Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Models (NeurIPS 2025 · similarity 0.87)
- Leveraging unlabelled data for generalizable neural population decoding (NeurIPS 2026 · similarity 0.87)
- Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.