Research map / Detection and segmentation / Open-set and few-shot detection
Decoding Natural Images from EEG for Object Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NICE, a self-supervised contrastive EEG-image framework (TSConv EEG encoder with self/graph attention modules and frozen CLIP image encoder) for 200-way zero-shot object recognition from EEG, with neuroscientific analyses.
Problem
Decoding natural object categories from non-invasive EEG is hard due to low signal-to-noise ratio, block-design confounds, and limited category coverage; prior supervised approaches generalize poorly to unseen classes.
Approach
Align EEG features with frozen pre-trained image features (CLIP) via symmetric InfoNCE contrastive learning, so unseen concepts can be classified zero-shot by matching EEG to averaged image templates; adds self-attention and graph-attention spatial modules over electrodes.
Key results
- Subject-dependent 200-way zero-shot: NICE-GA reaches 15.6% top-1 and 42.8% top-5 average, versus BraVL 5.8% top-1 and 17.5% top-5.
- Subject-independent (leave-one-out) NICE achieves 6.2% top-1 and 21.4% top-5 average.
- With EVA-CLIP as image encoder, NICE reaches 18.7% top-1 and 47.6% top-5 average (NICE base variant).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace BraVL with NICE | EEG decoding · 200-way zero-shot, subject-dependent | top-1 accuracy: 5.8 -> 13.8 | Helped |
| add self-attention (ablation) | NICE · EEG 200-way zero-shot, subject-dependent | top-1 accuracy: 13.8 -> 14.7; 0.9% | Helped |
| add graph attention (ablation) | NICE · EEG 200-way zero-shot, subject-dependent | top-1 accuracy: 13.8 -> 15.6; 1.8% | Helped |
| add self-attention (ablation) | NICE · EEG 200-way zero-shot, subject-independent | top-1 accuracy: 6.2 -> 7.0; 0.8% | No clear effect |
| add graph attention (ablation) | NICE · EEG 200-way zero-shot, subject-independent | top-1 accuracy: 6.2 -> 5.9; -0.3% | Hurt |
| use TSConv instead of EEGNet | NICE EEG encoder · Subject-dependent EEG decoding | top-1 accuracy: 13.1 -> 13.8; 0.7% | No clear effect |
| use CLIP instead of ResNet image encoder | NICE · Subject-dependent EEG decoding | top-1 accuracy: 8.8 -> 13.8; 6.9% | Helped |
| use pre-trained ViT-B/16 instead of non-pre-trained ViT-B/16 (ablation) | NICE image encoder · Subject-dependent EEG decoding | top-1 accuracy: 7.9 -> 12.1; 4.2% | Helped |
| use pre-trained ResNet-50 instead of non-pre-trained ResNet-50 (ablation) | NICE image encoder · Subject-dependent EEG decoding | top-1 accuracy: 8.8 -> 6.9; 1.9% | Hurt |
| use EVA-CLIP image encoder instead of CLIP | NICE · EEG 200-way zero-shot, subject-dependent | top-1 accuracy: 13.8 -> 18.7 | Helped |
| remove residual connection from graph attention (ablation) | NICE-GA · EEG 200-way zero-shot, subject-dependent | top-1 accuracy: 15.6 -> 10.1 | Hurt |
| remove residual connection from self-attention (ablation) | NICE-SA · EEG 200-way zero-shot, subject-dependent | top-1 accuracy: 14.7 -> 13.8 | No clear effect |
Surprising findings
- Frozen ImageNet-1k pre-trained ResNet-50 features decreased accuracy relative to the non-pretrained ResNet-50 (6.9% vs 8.8% top-1).
- Gamma band yielded little decoding performance, and the first 100 ms post-stimulus contained limited information.
Limitations
- Stable responses required many repetitions of the same stimulus; 100 ms stimulus duration may be too brief.
- Gamma-band information was not captured.
- Subject-independent performance is substantially lower than subject-dependent performance.
Experimental setup
Datasets: THINGS-EEG THINGS-MEG
Benchmarks: THINGS-EEG 200-way zero-shot THINGS-MEG 200-way zero-shot
Metrics: top-1 accuracy top-5 accuracy Wilcoxon signed-rank test
Baselines: BraVL ShallowNet DeepNet Conformer EEGNet ResNet-50 ViT-B/16
Builds on: CLIP InfoNCE EEGNet ShallowFBCSPNet
Closest papers by meaning
- What Does the Brain See? Multiview Neural Representation to Demystify the Brain-Visual Alignment (NeurIPS 2026 · similarity 0.91)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.91)
- Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion (NeurIPS 2024 · similarity 0.89)
- EEG2Video: Towards Decoding Dynamic Visual Perception from EEG Signals (NeurIPS 2024 · similarity 0.89)
- Neuro-3D: Towards 3D Visual Decoding from EEG Signals (CVPR 2025 · similarity 0.89)
- BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.