Research map / Vision-language and multimodal / Human and embodied modeling
Mind Reader: Reconstructing complex images from brain activities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| start from language-free Lafite instead of supervised Lafite (ablation) | Mind Reader f_mi · NSD val | FID: 37.75 -> 30.83 | Helped |
| add low-level contrastive loss L_c3 (ablation) | Mind Reader f_mi · NSD val | FID: 30.83 -> 29.74 | Helped |
| add L_c3 (ablation) | Mind Reader f_mi & f_mc · NSD val | FID: 50.59 -> 49.47 | Helped |
| add L_c3 (ablation) | Mind Reader f_mi · NSD val | 2-way identification accuracy (%): 73.0 -> 76.8 | Helped |
| finetune mapper end to end with GAN instead of frozen (ablation) | Mind Reader f_mi · NSD val | FID: 29.74 -> 45.02 | Hurt |
| finetune mapper end to end instead of frozen (ablation) | Mind Reader f_mi · NSD val | 2-way identification accuracy (%): 76.8 -> 51.4 | Hurt |
| use image-embedding mapper f_mi instead of caption mapper f_mc as condition (ablation) | Mind Reader · NSD val | FID: 33.35 -> 29.74 | Helped |
| use two conditional heads instead of one (ablation) | Mind Reader · NSD val | FID: 29.74 -> 49.47 | Hurt |
| clamp ground-truth CLIP embeddings with threshold at 1.5 (ablation) | f_mi mapper · fMRI-CLIP retrieval | backward retrieval correct: 25 -> 49 | Helped |
| use image augmentation before CLIP encoder (ablation) | f_mi mapper · fMRI-CLIP retrieval | backward retrieval correct: 46 -> 49 | Helped |
| use random valid caption instead of fixed caption (ablation) | f_mc mapper · fMRI-CLIP retrieval | backward retrieval correct: 39 -> 64 | Helped |
| finetune with MSE+cos+Contra after MSE+cos (ablation) | f_mc mapper · starting FID | FID: 75.24 -> 53.68 | Helped |
Closest papers by meaning
- See Through Their Minds: Learning Transferable Brain Decoding Models from Cross-Subject fMRI (AAAI 2025 · similarity 0.92)
- MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data (ICML 2024 · similarity 0.91)
- Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction (NeurIPS 2024 · similarity 0.90)
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.90)
- NSD-Imagery: A Benchmark Dataset for Extending fMRI Vision Decoding Methods to Mental Imagery (CVPR 2025 · similarity 0.90)
- MindBridge: A Cross-Subject Brain Decoding Framework (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.