AI Paper Atlas

Research map / Image generation / Autoregressive and latent generation

Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors

Paul Steven Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Cohen Ethan, Aidan James Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, Tanishq Mathew Abraham

NeurIPS 2023 · Image generation › Autoregressive and latent generation · approach: High-resolution diffusion synthesis · 40 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add diffusion prior to MLP backbone + projector (ablation)MindEye · NSD Subject 1CLIP two-way identification: 77.3% -> 95.0%Improved
add separate MLP projector for contrastive loss (ablation)MindEye retrieval · NSD Subject 1image retrieval: 0.934 -> 0.972Improved
add projector (decoupling losses) to backbone-only model (ablation)MLP backbone · NSD Subject 1image retrieval: 0.133 -> 0.888Improved
replace InfoNCE with bidirectional CLIP loss (ablation)MindEye retrieval · NSD Subject 1image retrieval: 0.237 -> 0.837Improved
add MixCo augmentation to CLIP loss (BiMixCo) (ablation)CLIP loss · NSD Subject 1image retrieval: 0.837 -> 0.884Improved
combine BiMixCo with SoftCLIP (ablation)BiMixCo · NSD Subject 1image retrieval: 0.884 -> 0.896Improved
switch from BiMixCo to SoftCLIP after one-third of training (ablation)MindEye · NSD Subject 1CLIP two-way identification: 94.4% -> 95.0%Improved
add skip connections to 4-block MLP (ablation)MLP backbone · NSD Subject 1image retrieval: 0.836 -> 0.896Improved
map to all 257 CLIP tokens instead of CLS only (7x more parameters) (ablation)MLP backbone · NSD Subject 1image retrieval: 0.611 -> 0.896Improved
add residual blocks (0 to 4) to MLP backbone (ablation)MLP backbone · NSD Subject 1image retrieval: 0.880 -> 0.896Improved
use high-level pipeline only (drop low-level img2img init) (ablation)MindEye · NSD testPixCorr: .309 -> .194Hurt
MindEye vs Ozcelik et al. on SSIMMindEye · NSD testSSIM: .356 -> .323Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy