Research map / Architectures and efficiency / Attention and transformers
BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BaRISTA, a spatiotemporal transformer for multiregional iEEG with a masked latent reconstruction pretraining task that decouples spatial encoding scale (channel, parcel, lobe) from masking scale, showing parcel-level encoding improves language decoding over channel-level and SOTA iEEG models.
Problem
Existing iEEG neurofoundation models encode space only at the single-channel level or ignore it, and use random channel-level masking; it is unclear how spatial scale for token encoding and masking affects self-supervised pretraining and downstream decoding of multiregional intracranial recordings.
Approach
A transformer that keeps channel-level temporal tokens but adds learnable spatial embeddings at chosen scales (channel LPI coordinates, atlas parcels, or lobes), interleaved with time tokens, plus a latent masked-token reconstruction task whose masks are defined by the same spatial meta-information, allowing encoding and masking scales to be varied independently.
Key results
- Parcel-level token encoding with channel-level masking reached AUC 0.862 on sentence onset and 0.869 on speech/non-speech, versus 0.795 and 0.775 for PopT, with significant differences (Wilcoxon p<1e-5 for the PopT comparison on both tasks).
- Channel-level spatial encoding underperformed parcel and lobe encoding across masking strategies; spatial encoding had a larger effect than spatial masking (two-way ANOVA).
- Parcel-level encoded models achieved masked channel reconstruction MSE of 0.354 to 0.413 and R^2 of about 0.59 to 0.65, comparable to channel-level encoding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use parcel-level encoding with channel-level masking instead of channel-level encoding and masking (ablation) | BaRISTA · Sentence Onset | ROC-AUC: 0.778 ± 0.019 -> 0.862 ± 0.016 | Helped |
| Use parcel-level encoding with channel-level masking instead of channel-level encoding and masking (ablation) | BaRISTA · Speech/Non-Speech | ROC-AUC: 0.764 ± 0.020 -> 0.869 ± 0.016 | Helped |
| Use lobe-level encoding instead of channel-level encoding (ablation) | BaRISTA · Sentence Onset and Speech/Non-Speech | ROC-AUC | Helped |
| Use parcel-level encoding instead of channel-level encoding (ablation) | BaRISTA · Sentence Onset and Speech/Non-Speech | ROC-AUC | Helped |
| Use parcel-level masking instead of channel-level masking while retaining channel-level encoding (ablation) | BaRISTA · Sentence Onset and Speech/Non-Speech | ROC-AUC: 0.778 ± 0.019 -> 0.710 ± 0.017; 0.764 ± 0.020 -> 0.668 ± 0.015 | Hurt |
| Use lobe-level masking instead of channel-level masking while retaining channel-level encoding (ablation) | BaRISTA · Sentence Onset and Speech/Non-Speech | ROC-AUC: 0.778 ± 0.019 -> 0.654 ± 0.019; 0.764 ± 0.020 -> 0.652 ± 0.017 | Hurt |
| Use parcel-level masking instead of channel-level masking while retaining parcel-level encoding (ablation) | BaRISTA · Sentence Onset and Speech/Non-Speech | ROC-AUC: 0.862 ± 0.016 -> 0.861 ± 0.014; 0.869 ± 0.016 -> 0.866 ± 0.015 | No clear effect |
| Pretrain BaRISTA instead of using random initialization (ablation) | BaRISTA with channel-level encoding · Sentence Onset | ROC-AUC: 0.688 ± 0.017 -> 0.778 ± 0.019 | Helped |
| Pretrain BaRISTA instead of using random initialization (ablation) | BaRISTA with parcel-level encoding · Speech/Non-Speech | ROC-AUC: 0.627 ± 0.018 -> 0.869 ± 0.016 | Helped |
| Use parcel-level spatial encoding rather than channel-level spatial encoding (ablation) | BaRISTA · Masked channel reconstruction | MSE: 0.397 ± 0.040 -> 0.391 ± 0.019 | No clear effect |
| Hold out the target subject during pretraining (ablation) | BaRISTA (parcels/channels) · Sentence Onset | ROC-AUC: 0.862 ± 0.015 -> 0.841 ± 0.016 | Hurt |
| Hold out the target subject during pretraining (ablation) | BaRISTA (parcels/channels) · Speech/Non-Speech | ROC-AUC: 0.869 ± 0.016 -> 0.852 ± 0.013 | Hurt |
Surprising findings
- Channel-level encoding underperformed larger-scale encoding regardless of masking scale, even though the model is much smaller than PopT and Brant.
- Spatial encoding choice had no measurable effect in the randomly initialized setting.
Limitations
- null
Experimental setup
Datasets: Brain Treebank
Benchmarks: Brain Treebank sentence onset classification Brain Treebank speech/non-speech classification masked channel reconstruction
Metrics: ROC-AUC MSE R^2
Baselines: Brant PopT PopT+Brainbert random initialization
Builds on: JEPA-style latent prediction Population Transformer Brant RoPE
Closest papers by meaning
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.88)
- Neural Encoding and Decoding at Scale (ICML 2025 · similarity 0.87)
- NeurIPT: Foundation Model for Neural Interfaces (NeurIPS 2025 · similarity 0.87)
- Dynamical modeling of nonlinear latent factors in multiscale neural activity with real-time inference (NeurIPS 2025 · similarity 0.87)
- Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals (NeurIPS 2024 · similarity 0.87)
- Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.