Research map / 3D vision / 3D generation
Neuro-3D: Towards 3D Visual Decoding from EEG Signals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EEG-3D, a 12-subject EEG dataset for 72 categories of rotating 3D objects, and Neuro-3D, a dynamic-static EEG fusion encoder with a diffusion-based colored point cloud decoder that reconstructs 3D shape and color from EEG.
Problem
Decoding 3D visual perception (shape and color of real-world 3D objects) from non-invasive EEG signals, which previously lacked paired EEG and 3D stimulus data.
Approach
Adaptively fuse static-image and rotating-video EEG features with an attention-based aggregator, decouple the fused embedding into geometry and appearance features, and decode shape via a diffusion point cloud model followed by a single-step color model.
Key results
- Neuro-3D reaches 5.91% top-1 and 16.30% top-5 on 72-way object classification, above the best baseline TSConv (4.05%/10.13%).
- On color type classification Neuro-3D reaches 39.93% top-1 vs 31.13% for TSConv.
- Full 3D reconstruction achieves 2-way top-1 of 72.08 and 10-way top-3 of 57.64 (averaged), Chamfer Distance 5.35 and F1 77.01, improving over concatenation fusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the dynamic-static EEG-fusion encoder instead of Conformer | EEG visual classification · Object classification (72 categories) | top-1 accuracy (%): 4.05 -> 5.91 | Helped |
| Use the dynamic-static EEG-fusion encoder instead of TSConv | EEG visual classification · Color type classification (6 categories) | top-1 accuracy (%): 31.13 -> 39.93 | Helped |
| Add static EEG features to dynamic EEG features (ablation) | Neuro-3D classification encoder · Object classification (72 categories) | top-1 accuracy (%): 4.75 -> 5.91 | Helped |
| Add dynamic EEG features to static EEG features (ablation) | Neuro-3D classification encoder · Object classification (72 categories) | top-1 accuracy (%): 5.10 -> 5.91 | Helped |
| Replace separate static and dynamic features with their concatenation, without aggregation (ablation) | Neuro-3D classification encoder · Object classification (72 categories) | top-1 accuracy (%): 5.44 -> 5.91 | Hurt |
| Use only static EEG features instead of the full method (ablation) | Neuro-3D classification encoder · Color type classification (6 categories) | top-1 accuracy (%): 37.50 -> 39.93 | Hurt |
| Use only dynamic EEG features instead of the full method (ablation) | Neuro-3D classification encoder · Color type classification (6 categories) | top-1 accuracy (%): 35.65 -> 39.93 | Hurt |
| Use only dynamic EEG features instead of fused static and dynamic features (ablation) | Neuro-3D reconstruction pipeline · 3D object reconstruction | Chamfer Distance (×10−2): 6.40 -> 5.35 | Hurt |
| Use only static EEG features instead of fused static and dynamic features (ablation) | Neuro-3D reconstruction pipeline · 3D object reconstruction | Chamfer Distance (×10−2): 6.75 -> 5.35 | Hurt |
| Replace adaptive aggregation with static-dynamic feature concatenation (ablation) | Neuro-3D reconstruction pipeline · 3D object reconstruction | F1 score (%): 73.47 -> 77.01 | Hurt |
| Use non-decoupled EEG features for shape and color (ablation) | Neuro-3D reconstruction pipeline · 3D object reconstruction | F1 score (%): 73.36 -> 77.01 | Hurt |
Surprising findings
- Removing occipital electrodes hurts performance most, while temporal or parietal removal hurts dynamic stimuli more than static, consistent with the dorsal visual pathway.
Limitations
- Color is simplified to dominant color style prediction rather than full texture generation.
- Substantial individual variability in EEG limits cross-subject generalization.
- Low EEG signal-to-noise ratio and spatial resolution limit detailed reconstruction.
Experimental setup
Datasets: EEG-3D Objaverse
Benchmarks: EEG-3D 72-way object classification EEG-3D 6-way color type classification EEG-3D 3D object reconstruction
Metrics: top-1 accuracy top-5 accuracy N-way top-K accuracy Chamfer Distance F1 score
Baselines: DeepNet EEGNet Conformer TSConv
Builds on: Mind-3D Point-Voxel Network CLIP Zero-123 Objaverse
Closest papers by meaning
- EEG2Video: Towards Decoding Dynamic Visual Perception from EEG Signals (NeurIPS 2024 · similarity 0.90)
- What Does the Brain See? Multiview Neural Representation to Demystify the Brain-Visual Alignment (NeurIPS 2026 · similarity 0.90)
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.89)
- Decoding Natural Images from EEG for Object Recognition (ICLR 2024 · similarity 0.89)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.88)
- Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.