Research map / Vision-language and multimodal / Human and embodied modeling
Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features
Simone Palazzo, Concetto Spampinato, Isaak Kavasidis, Daniela Giordano, Joseph Schmidt, Mubarak Shah
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| restrict EEG input to high gamma band 55-95 Hz instead of all frequencies (ablation) | EEG-ChannelNet · EEG classification | accuracy: 31.3% -> 48.1% | Improved |
| use high gamma instead of all gamma (ablation) | EEG-ChannelNet · EEG classification | accuracy: 40.0% -> 48.1% | Improved |
| use entire 20-460 ms EEG course instead of 20-350 ms (ablation) | EEG-ChannelNet · EEG classification | accuracy: 43.8% -> 48.1% | Improved |
| include the first 110 ms of EEG (20-460 vs 130-460) (ablation) | EEG-ChannelNet · EEG classification | accuracy: 26.3% -> 48.1% | Improved |
| add joint visual-EEG embedding learning (ablation) | EEG-ChannelNet · EEG classification | accuracy: 48.1% -> 60.4% | Improved |
| add joint EEG-visual fine-tuning (ablation) | Inception-v3 · Image classification | accuracy: 93.1% -> 94.4% | Improved |
| add joint EEG-visual fine-tuning (ablation) | ResNet-101 · Image classification | accuracy: 90.3% -> 90.5% | Improved |
| add joint EEG-visual fine-tuning (ablation) | DenseNet-161 · Image classification | accuracy: 91.4% -> 92.1% | Improved |
| add joint EEG-visual fine-tuning (ablation) | AlexNet · Image classification | accuracy: 65.5% -> 69.4% | Improved |
| use EEG-visual compatibility instead of classifier log-likelihood for saliency | patch suppression saliency · Saliency detection | NSS: 0.495 -> 0.942 | Improved |
| use neural-driven saliency instead of SALICON | SALICON · Saliency detection | s-AUC: 0.678 -> 0.643 | Hurt |
| use neural-driven saliency instead of SALICON | SALICON · Saliency detection | NSS: 0.728 -> 0.942 | Improved |
Closest papers by meaning
- Decoding Natural Images from EEG for Object Recognition (ICLR 2024 · similarity 0.89)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.89)
- What Does the Brain See? Multiview Neural Representation to Demystify the Brain-Visual Alignment (NeurIPS 2026 · similarity 0.89)
- Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion (NeurIPS 2024 · similarity 0.88)
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.88)
- Neuro-3D: Towards 3D Visual Decoding from EEG Signals (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
