Research map / Vision-language and multimodal / Human and embodied modeling
Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Uncertainty-aware Blur Prior (UBP), which blurs stimulus images adaptively based on a confidence-interval-derived uncertainty of paired EEG/MEG-image similarity, to improve contrastive vision-brain alignment for zero-shot brain-to-image retrieval.
Problem
Brain signals (EEG/MEG) do not faithfully reflect visual stimuli (System GAP and Random GAP), and direct contrastive alignment with pretrained image features overfits under limited paired data, hurting zero-shot brain-to-image retrieval.
Approach
Apply a foveated Gaussian blur to the image side of vision-brain contrastive learning, with per-sample blur radius adjusted by whether the paired similarity falls outside a Gaussian-estimated confidence interval, so high-uncertainty (mismatched) pairs get more blur.
Key results
- UBP reaches 50.9% top-1 and 79.7% top-5 for 200-way zero-shot brain-to-image retrieval on THINGS-EEG intra-subject, versus 37.2% and 69.9% for VE-SDN.
- On THINGS-MEG intra-subject, UBP achieves 26.7% top-1 and 55.2% top-5, versus 14.3% and 42.3% for NICE-GA.
- Uniform blur at radius 11 gives the best top-1 among uniform blur radii in the sensitivity study, and fovea blur plus uncertainty (UBP) improves over fixed blur variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use UBP instead of VE-SDN | vision-brain decoding · THINGS-EEG intra-subject | top-1 accuracy: 37.2 -> 50.9 | Helped |
| use UBP instead of VE-SDN | vision-brain decoding · THINGS-EEG intra-subject | top-5 accuracy: 69.9 -> 79.7 | Helped |
| use UBP instead of ATM-S | vision-brain decoding · THINGS-EEG inter-subject | top-1 accuracy: 11.8 -> 12.4 | Helped |
| use UBP instead of ATM-S | vision-brain decoding · THINGS-EEG inter-subject | top-5 accuracy: 33.7 -> 33.4 | Hurt |
| use UBP instead of NICE-GA | vision-brain decoding · THINGS-MEG intra-subject | top-1 accuracy: 14.3 -> 26.7 | Helped |
| use UBP instead of NICE-GA | vision-brain decoding · THINGS-MEG intra-subject | top-5 accuracy: 42.3 -> 55.2 | Helped |
| add blur prior that reduces high-frequency image detail (ablation) | vision-brain decoding · THINGS-EEG | top-1 accuracy: 42.1 -> 49.3 | Helped |
| replace uniform blur with foveated blur (ablation) | vision-brain decoding · THINGS-EEG intra-subject | top-1 accuracy: 49.3 -> 50.2 | Helped |
| add uncertainty-aware dynamic blur to foveated blur (ablation) | vision-brain decoding · THINGS-EEG intra-subject | top-1 accuracy: 50.2 -> 50.9 | Helped |
| use blur radius 11 rather than no blur (ablation) | vision-brain decoding · THINGS-EEG | top-1 accuracy | Helped |
| increase blur beyond the optimal radius (ablation) | vision-brain decoding · THINGS-EEG | top-1 accuracy | Hurt |
Surprising findings
- Simple image corruptions that remove high-frequency detail (noise, low-resolution, blur) improve retrieval more than geometric or color augmentations.
- Excessive blur (radius 41) yields worse performance than no blur, though moderate blur helps.
Limitations
- Blur prior is a simplified approximation of the human visual system's high-frequency loss and lacks completeness.
- Uncertainty quantification may fail due to the complexity of Random GAP from perception, cognition, and noise.
- Inter-subject performance remains low (avg top-1 12.4% on THINGS-EEG).
Experimental setup
Datasets: THINGS-EEG THINGS-MEG
Benchmarks: THINGS-EEG THINGS-MEG
Metrics: top-1 accuracy top-5 accuracy mAP similarity score Pearson correlation Spearman correlation
Baselines: BraVL NICE NICE-SA NICE-GA ATM-S VE-SDN
Builds on: CLIP NICE symmetric cross-entropy loss
Closest papers by meaning
- Neural‑Visual Decoding via Cognitive‑guided Adaptive Blurring and Information‑Constrained Alignment (NeurIPS 2026 · similarity 0.88)
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.86)
- Human-Aligned Image Models Improve Visual Decoding from the Brain (ICML 2025 · similarity 0.86)
- Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels (NeurIPS 2025 · similarity 0.85)
- Multigranular Evaluation for Brain Visual Decoding (AAAI 2026 · similarity 0.85)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.