Research map / Image generation / Text-to-image diffusion
DreamDiffusion: High-Quality EEG-to-Image Generation with Temporal Masked Signal Modeling and CLIP Alignment
Yunpeng Bai, Xintao Wang, Yan-Pei Cao, Yixiao Ge, Chun Yuan, Ying Shan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add masked signal modeling pretraining on large EEG data (ablation) | DreamDiffusion · ImageNet-EEG 50-way top-1 | accuracy (%): 32.3 -> 45.8 | Improved |
| add CLIP image-embedding alignment during fine-tuning (ablation) | DreamDiffusion · ImageNet-EEG 50-way top-1 | accuracy (%): 28.3 -> 45.8 | Improved |
| add CLIP supervision without pretraining (ablation) | unpretrained EEG encoder · ImageNet-EEG 50-way top-1 | accuracy (%): 4.2 -> 32.3 | Improved |
| use mask ratio 0.75 instead of 0.25 (ablation) | MSM pretraining · ImageNet-EEG 50-way top-1 | accuracy (%): 19.7 -> 45.8 | Improved |
| increase mask ratio from 0.75 to 0.85 (ablation) | MSM pretraining · ImageNet-EEG 50-way top-1 | accuracy (%): 45.8 -> 33.4 | Hurt |
| increase EEG encoder size to 458M parameters from 297M (ablation) | EEG encoder · ImageNet-EEG 50-way top-1 | accuracy (%): 45.8 -> 38.5 | Hurt |
| shrink EEG encoder from 297M to 18.3M parameters (ablation) | EEG encoder · ImageNet-EEG 50-way top-1 | accuracy (%): 45.8 -> 28.7 | Hurt |
| fine-tune both EEG encoder and cross-attention heads instead of encoder only (ablation) | DreamDiffusion · ImageNet-EEG 50-way top-1 | accuracy (%): 22.4 -> 45.8 | Improved |
| fine-tune both encoder and cross-attention instead of attention only (ablation) | DreamDiffusion · ImageNet-EEG 50-way top-1 | accuracy (%): 20.9 -> 45.8 | Improved |
| DreamDiffusion versus Brain2Image | Brain2Image · ImageNet-EEG (Brain2Image categories) | FID: 18.76 -> 3.61 | Improved |
| DreamDiffusion versus Brain2Image on LPIPS and SSIM | Brain2Image · ImageNet-EEG (Brain2Image categories) | LPIPS: 0.701 -> 0.644 | Improved |
| supervise with category labels instead of CLIP | DreamDiffusion · ImageNet-EEG 50-way top-1 | accuracy (%): 45.8 -> 86.7 | Improved |
Closest papers by meaning
- Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion (NeurIPS 2024 · similarity 0.87)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.86)
- CognitionCapturer: Decoding Visual Stimuli from Human EEG Signal with Multimodal Information (AAAI 2025 · similarity 0.85)
- EEG2Video: Towards Decoding Dynamic Visual Perception from EEG Signals (NeurIPS 2024 · similarity 0.85)
- What Does the Brain See? Multiview Neural Representation to Demystify the Brain-Visual Alignment (NeurIPS 2026 · similarity 0.85)
- Decoding Natural Images from EEG for Object Recognition (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
