Research map / Image generation / Autoregressive and latent generation
Seeing Through the Brain: New Insights from Decoding Visual Stimuli with fMRI
Zheng Huang, Enpei Zhang, Weikang Qiu, Yinghao Cai, Carl Yang, Elynn Chen, Xiang Zhang, Rex Ying, Dawei Zhou, Yujun Yan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add object-centric cross-attention (ablation) | fMRI-to-image framework · NSD | LPIPS: 0.6111 -> 0.5963 | Improved |
| add object-centric cross-attention (ablation) | fMRI-to-image framework · NSD | SSIM: 0.4299 -> 0.4640 | Improved |
| add attribute/relationship prompt search instead of best initial keyword (ablation) | fMRI-to-image framework · NSD | SSIM: 0.4421 -> 0.4640 | Improved |
| add prompt search instead of worst initial keyword (ablation) | fMRI-to-image framework · NSD | LPIPS: 0.6398 -> 0.5963 | Improved |
| use LM text space as latent instead of CLIP text (ablation) | fMRI-to-image framework · NSD | LPIPS: 0.6611 -> 0.5963 | Improved |
| use LM text space as latent instead of LDM image latent (ablation) | fMRI-to-image framework · NSD | LPIPS: 0.7502 -> 0.5963 | Improved |
| use two objects instead of one (ablation) | fMRI-to-image framework · NSD | LPIPS: 0.6014 -> 0.5943 | Improved |
| use two objects instead of four (ablation) | fMRI-to-image framework · NSD | LPIPS: 0.6284 -> 0.5943 | Improved |
| swap Stable Diffusion 2.1 backbone for SDXL | fMRI-to-image framework · NSD | LPIPS: 0.5963 -> 0.5563 | Improved |
| compare T5 text space to ResNet50 vision space for fMRI alignment | representation spaces · fMRI-representation alignment | CKA: 0.1822 -> 0.5580 | Improved |
| compare T5 text space to ResNet50 for learnability | representation spaces · fMRI-representation alignment | Generalization Gap: 1.9800 -> 0.1132 | Improved |
| use the method instead of Mindeye2 for QA on reconstructions | fMRI decoders · NSD image QA | accuracy: 0.5765 -> 0.6054 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Mindeye2 — MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data (ICML 2024) · better on 8 of 8 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.91)
- MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data (ICML 2024 · similarity 0.91)
- See Through Their Minds: Learning Transferable Brain Decoding Models from Cross-Subject fMRI (AAAI 2025 · similarity 0.90)
- Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction (NeurIPS 2024 · similarity 0.90)
- SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning (NeurIPS 2025 · similarity 0.90)
- NSD-Imagery: A Benchmark Dataset for Extending fMRI Vision Decoding Methods to Mental Imagery (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
