Research map / Vision-language and multimodal / CLIP and zero-shot

MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding

NeurIPS 2026 · Vision-language and multimodal › CLIP and zero-shot · approach: Language models for zero-shot reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a two-stage tri-modal EEG–image–text contrastive framework (MAE-style EEG pre-training, then alignment with frozen CLIP image/text encoders and LLM-generated captions) for zero-shot visual decoding on Things-EEG2 and Things-MEG.

Problem

Zero-shot visual decoding from low-SNR, high inter-subject-variability EEG is limited by weak EEG–image supervision; prior work neglects structured linguistic semantics as complementary supervision.

Approach

Pre-train an EEG encoder (subject-specific layer, GAT over channels, Transformer, channel attention with electrode priors, TSConv patch embedding) by masked Gaussian-noise reconstruction, then jointly align EEG with frozen CLIP image embeddings and LLM-generated text descriptions via two InfoNCE losses weighted by a small alpha=0.1.

Key results

• On Things-EEG2 in-subject 200-way zero-shot, achieves 54.1% Top-1 and 83.4% Top-5 vs. strongest prior 32.4%/64.0%.
• On Things-MEG in-subject, average 25.3% Top-1 / 53.9% Top-5 vs. NICE++ 11.8% / 32.4%.
• CN-CLIP (RN50, 38M) gives 54.1% Top-1 vs. 37.4% for CLIP ViT-G-14 (1.37B) in the ablation.

Surprising findings

• A small ResNet-based CN-CLIP backbone outperforms much larger CLIP ViT backbones for EEG-to-image retrieval.
• Larger text-alignment weight (alpha=0.5) drops image Top-1 below the text-free baseline.

Limitations

• Cross-subject Top-1 (~15% average) remains far below in-subject performance.
• Evaluation limited to THINGS-based datasets with 200-way retrieval.
• Frozen CLIP encoders and reliance on LLM-generated captions from category labels.

Experimental setup

Datasets: Things-EEG2 Things-MEG

Benchmarks: Things-EEG2 Things-MEG

Metrics: Top-1 accuracy Top-5 accuracy Top-3 accuracy

Baselines: NICE NICE++ ATMS MCRL UBP ENIGMA

Builds on: CLIP Masked Autoencoder (MAE) NICE ATMS MCRL InfoNCE

Closest papers by meaning

• What Does the Brain See? Multiview Neural Representation to Demystify the Brain-Visual Alignment (NeurIPS 2026 · similarity 0.94)
• Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion (NeurIPS 2024 · similarity 0.92)
• Neural‑Visual Decoding via Cognitive‑guided Adaptive Blurring and Information‑Constrained Alignment (NeurIPS 2026 · similarity 0.91)
• Decoding Natural Images from EEG for Object Recognition (ICLR 2024 · similarity 0.91)
• Human-Aligned Image Models Improve Visual Decoding from the Brain (ICML 2025 · similarity 0.90)
• Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction (NeurIPS 2024 · similarity 0.89)