Research map / Vision-language and multimodal / Human and embodied modeling
TRIBE: TRImodal Brain Encoder for whole-brain fMRI response prediction
Stéphane d'Ascoli, Jérémy Rapin, Yohann Benchetrit, Hubert Banville, Jean-Remi King
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add video modality to text-only encoder (single to multimodal) (ablation) | TRIBE · Validation encoding score | mean Pearson correlation: 0.22 -> 0.30 (text+video) | Improved |
| add third modality (audio) to best bimodal model (ablation) | TRIBE · Validation encoding score | mean Pearson correlation: 0.30 -> 0.31 | Improved |
| use multimodal instead of best unimodal encoder in primary visual cortex (ablation) | TRIBE · Primary visual cortex parcels | encoding score | Hurt |
| use multimodal instead of best unimodal encoder in associative cortices (ablation) | TRIBE · Prefrontal and parieto-occipito-temporal parcels | encoding score: up to 30% increase | Improved |
| train a multi-subject model instead of separate per-subject models (ablation) | TRIBE · Validation encoding score | mean Pearson correlation: 0.29 -> 0.31 | Improved |
| add the transformer encoder over time (ablation) | TRIBE · Validation encoding score | mean Pearson correlation: 0.23 -> 0.31 | Improved |
| increase number of training sessions (ablation) | TRIBE · Multimodal encoder scaling | encoding score | Improved |
| increase LLM context length for text features up to 1024 words (ablation) | TRIBE text-only · Text encoder scaling | encoding score | Improved |
| evaluate out-of-distribution instead of in-distribution (ablation) | TRIBE · Friends S7 vs OOD movies | mean Pearson correlation: 0.3195 -> 0.2146 | Hurt |
Closest papers by meaning
- MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding (NeurIPS 2026 · similarity 0.88)
- Learning Sparse Semantic-Cortical Atoms for Multisubject Naturalistic fMRI Encoding (NeurIPS 2026 · similarity 0.87)
- Scaling laws for language encoding models in fMRI (NeurIPS 2023 · similarity 0.87)
- Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex (NeurIPS 2025 · similarity 0.87)
- Brain Harmony: A Multimodal Foundation Model Unifying Morphology and Function into 1D Tokens (NeurIPS 2025 · similarity 0.86)
- Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Data (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
