Research map / Label-efficient and robust learning / Recognition and long-tail
Real-World Unsupervised Models Generalize to Predict Brain Responses to Out-of-Distribution Stimuli
Chenggang Chen, Zhiyu Yang, Xiaoqin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain on real-world WenetSpeech instead of curated LibriSpeech (ablation) | HuBERT/Wav2Vec2 Base · NH2015 auditory fMRI | explained variance: 0.773 vs previous best 0.729 | Improved |
| pretrain on small uncurated FSD50K+AudioSet (153h) instead of LibriSpeech (960h) (ablation) | HuBERT · B2021 auditory fMRI | RSA: 15.5% gain | Improved |
| fine-tune pretrained Wav2Vec2 on LibriSpeech (ablation) | Wav2Vec2LS · NH2015 / B2021 auditory fMRI | explained variance: no improvement on NH2015, deteriorated on B2021 | No clear effect |
| fine-tune pretrained Wav2Vec2 for speech (ablation) | Wav2Vec2 · NH2015 speech component | explained variance rank: ranked 6th | Improved |
| use unsupervised Mandarin-trained model instead of supervised English models | HuBERTspeech · NH2015 RSA | Spearman rho: 0.506 vs 0.491 | Improved |
| use HuBERTspeech instead of CochResNet50-MultiTask | global RDM alignment · NH2015 CKA | CKA: 0.7182 vs 0.7012 | Improved |
| use HuBERTspeech instead of CochResNet50-MultiTask | global RDM alignment · B2021 CKA | CKA: 0.7331 vs 0.7013 | Improved |
| use supervised ResNet50-MultiTask instead of HuBERTspeech | local neighborhood alignment · MkNN at small k | mutual kNN overlap: k < 40 NH2015; k < 10 B2021 | Mixed |
| train on infant head-cam video instead of ImageNet (curated) (ablation) | ResNeXt-50-32x4d · Brain-Score Bracci2019 | Brain-Score: 0.169 -> 0.249 to 0.259 | Improved |
| scale ImageNet ResNeXt depth/width (50-32x4d to 101-64x4d) (ablation) | ResNeXt on ImageNet · Brain-Score Bracci2019 | Brain-Score: 0.169, 0.196, 0.21 with ImageNet acc 77.6%, 79.3%, 83.2% | Mixed |
| train on infant head-cam video instead of web-curated images (DINO) | baby-vision ResNeXt · Brain-Score Bracci2019 | Brain-Score: 0.249 to 0.259 vs 0.251 to 0.262 | No clear effect |
| train on SAYCam head-cam unsupervised models vs previous leader | camS/camA/camY/camSAY · Brain-Score tong.Coggan2024 fMRI.IT | Brain-Score: 0.267 -> 0.438, 0.4167, 0.4137, 0.393 | Improved |
Closest papers by meaning
- Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex. (NeurIPS 2024 · similarity 0.89)
- Generalization in data-driven models of primary visual cortex (ICLR 2021 · similarity 0.87)
- TRIBE: TRImodal Brain Encoder for whole-brain fMRI response prediction (ICLR 2026 · similarity 0.87)
- Toward Generalizing Visual Brain Decoding to Unseen Subjects (ICLR 2025 · similarity 0.87)
- AND: Audio Network Dissection for Interpreting Deep Acoustic Models (ICML 2024 · similarity 0.86)
- MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
