Research map / Vision-language and multimodal / Human and embodied modeling
Neural encoding with visual attention
Meenakshi Khosla, Gia Ngo, Keith Jamison, Amy Kuceyeski, Mert Sabuncu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add gaze-weighted attention pooling (vs no attention), linear response model (ablation) | ResNet-50 encoder · HCP 7T fMRI movie test | mean prediction accuracy over synchronous voxels: ~ 40 \% improvement | Improved |
| add gaze-weighted attention pooling (vs no attention), convolutional response model (ablation) | ResNet-50 encoder · HCP 7T fMRI movie test | mean prediction accuracy over synchronous voxels: ~ 47 \% improvement | Improved |
| use convolutional instead of linear response model (ablation) | encoding model · HCP 7T fMRI movie test | prediction accuracy | Improved |
| replace center-weighted or no attention with learned soft attention (ablation) | convolutional encoding model · HCP 7T fMRI movie test | mean prediction accuracy | Improved |
| attention modulation on other ResNet-50 layers (ablation) | encoding models · HCP 7T fMRI movie test | prediction accuracy | Improved |
| learned attention vs Deepgaze-II on saliency | attention network · gaze prediction | SIM: 0.359 -> 0.392 | Improved |
| learned attention vs Deepgaze-II on saliency | attention network · gaze prediction | NSS: 1.425 -> 1.375 | Hurt |
| learned attention vs Deepgaze-II on saliency | attention network · gaze prediction | AUC: 0.808 -> 0.754 | Hurt |
| learned attention vs center-weighted baseline on saliency | attention network · gaze prediction | CC: 0.350 -> 0.403 | Improved |
| hemodynamic delay of 4 s chosen (ablation) | regression encoding · HCP 7T fMRI | prediction accuracy | Improved |
Closest papers by meaning
- Self-Attention-Based Contextual Modulation Improves Neural System Identification (ICLR 2025 · similarity 0.87)
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.87)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.87)
- Transformer brain encoders explain human high-level visual responses (NeurIPS 2025 · similarity 0.87)
- See Through Their Minds: Learning Transferable Brain Decoding Models from Cross-Subject fMRI (AAAI 2025 · similarity 0.87)
- Attention Alignment Between Humans and Vision-Language Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
