Research map / Label-efficient and robust learning / Recognition and long-tail
MindBridge: A Cross-Subject Brain Decoding Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MindBridge, a single-model cross-subject fMRI-to-image brain decoding framework using adaptive max pooling, cyclic fMRI reconstruction, and reset-tuning for new subjects.
Problem
Brain decoding is restricted to a per-subject-per-model paradigm because fMRI input sizes differ across subjects, neural responses are subject-specific, and new subjects have little training data.
Approach
Unify variable-size fMRI voxels with adaptive max pooling, learn subject-invariant semantic embeddings with a cyclic fMRI reconstruction loss (synthesizing one subject's signals from another's), map them to CLIP embeddings, and adapt to new subjects by resetting only the shallow subject-specific layers while freezing the translator.
Key results
- A single MindBridge model averaged across 4 subjects reaches PixCorr 0.151, CLIP 94.7%, close to the 4-model MindEye baseline (PixCorr 0.309, CLIP 94.1%) and comparable to MindBridge (Single) (PixCorr 0.148).
- With 500 new-subject samples, reset-tuning reaches PixCorr 0.112 versus 0.079 for training from scratch (Vanilla); with 4000 samples it reaches 0.156 vs 0.114.
- Adaptive max pooling outperforms adaptive average pooling and interpolation as aggregation (PixCorr 0.165 vs 0.163 and 0.151 on subject 1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use one cross-subject model instead of separate subject-specific models | MindBridge · NSD, average across 4 subjects | CLIP: 94.1% -> 94.7% | Helped |
| use pretrained cross-subject knowledge with reset-tuning rather than train from scratch | MindBridge, NSD subject 7 · 500 data points | CLIP: 80.1% -> 86.7% | Helped |
| use pretrained cross-subject knowledge with reset-tuning rather than train from scratch | MindBridge, NSD subject 7 · 1500 data points | CLIP: 87.2% -> 91.0% | Helped |
| use pretrained cross-subject knowledge with reset-tuning rather than train from scratch | MindBridge, NSD subject 7 · 4000 data points | CLIP: 89.8% -> 92.5% | Helped |
| replace interpolation with adaptive max pooling (ablation) | MindBridge · NSD subject 1 | CLIP: 94.4% -> 95.0% | Helped |
| replace adaptive average pooling with adaptive max pooling (ablation) | MindBridge · NSD subject 1 | CLIP: 94.5% -> 95.0% | Helped |
| add MSE loss to SoftCLIP loss (ablation) | MindBridge pretraining · NSD subject 1 | CLIP: 80.6% -> 94.5% | Helped |
| add reconstruction and cycle losses (ablation) | MindBridge pretraining · NSD subject 1 | CLIP: 94.5% -> 94.9% | Helped |
| use full-tuning with image/text losses only instead of reset-tuning with total loss (ablation) | MindBridge adaptation · NSD subject 7 | CLIP: 86.7% -> 86.4% | Hurt |
| use full-tuning with total loss instead of reset-tuning with total loss (ablation) | MindBridge adaptation · NSD subject 7 | CLIP: 86.7% -> 86.0% | Hurt |
| use reset-tuning with image/text losses only instead of reset-tuning with total loss (ablation) | MindBridge adaptation · NSD subject 7 | CLIP: 86.7% -> 86.1% | Hurt |
Surprising findings
- Reset-tuning, which freezes the brain translator and retrains only shallow subject-specific layers, outperformed full fine-tuning on new subjects.
- The single cross-subject model's PixCorr (0.151) is far below MindEye's (0.309), though its CLIP/Inception scores are comparable.
Limitations
- Evaluated only on the small NSD dataset with four subjects.
- fMRI signals are serialized into 1D vectors, possibly losing spatial structure.
- Cross-subject MindBridge's low-level metrics (PixCorr, SSIM) lag behind MindEye.
Experimental setup
Datasets: Natural Scenes Dataset (NSD)
Benchmarks: Natural Scenes Dataset (NSD) 982-image shared test set
Metrics: PixCorr SSIM AlexNet(2) AlexNet(5) Inception CLIP EffNet-B SwAV
Baselines: Takagi et al. Brain-Diffuser MindEye MindBridge (Single) Vanilla MindBridge
Builds on: MindEye Versatile Diffusion SoftCLIP loss Brain-Diffuser
Closest papers by meaning
- MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-text Decoding (ICML 2025 · similarity 0.91)
- MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data (ICML 2025 · similarity 0.91)
- See Through Their Minds: Learning Transferable Brain Decoding Models from Cross-Subject fMRI (AAAI 2025 · similarity 0.90)
- $i$MIND: Insightful Multi-subject Invariant Neural Decoding (NeurIPS 2025 · similarity 0.90)
- Toward Generalizing Visual Brain Decoding to Unseen Subjects (ICLR 2025 · similarity 0.89)
- MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.