Research map / Vision-language and multimodal / Cross-modal learning
CineBrain: A Large-Scale Multi-Modal Audiovisual Brain Dataset for Brain-Conditioned Video Generation
Jianxiong Gao, Yichang Liu, Baofeng Yang, Jianfeng Feng, Yanwei Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fuse fMRI and EEG (CineSync) instead of fMRI only (ablation) | CineSync · CineBrain test | FVD: 57.47 -> 52.78 | Improved |
| fuse fMRI and EEG instead of EEG only (ablation) | CineSync · CineBrain test | video 50-way accuracy: 0.304 -> 0.319 | Improved |
| add auditory-cortex ROIs to fMRI input (ablation) | CineSync · CineBrain test | FVD: 52.78 -> 44.77 | Improved |
| add auditory-cortex ROIs to fMRI input (ablation) | CineSync · CineBrain test | PSNR: 11.99 -> 12.18 | Improved |
| add auditory-cortex ROIs to fMRI input (ablation) | CineSync · CineBrain test | CTC: 0.967 -> 0.953 | Hurt |
| encode fMRI and EEG with separate transformers (dual) instead of joint transformer (ablation) | Multi-Modal Fusion Encoder · CineBrain test | FVD: 128.0 -> 51.53 | Improved |
| cross-attention fusion instead of dual transformer fusion (ablation) | Multi-Modal Fusion Encoder · CineBrain test | FVD: 51.53 -> 110.0 | Hurt |
| allocate more tokens to EEG in spatial concatenation (f34-e192 vs f113-e113) (ablation) | Spatial concatenation · CineBrain test | 50-way accuracy: 0.276 -> 0.311 | Improved |
| use CineSync-EEG instead of EEG2Video | EEG decoding · CineBrain test | FVD: 146.23 -> 53.75 | Improved |
| use CineSync-fMRI instead of NeuroClips | fMRI decoding · CineBrain test | FVD: 116.36 -> 57.47 | Improved |
| add multi-level contrastive alignment (ablation) | CineSync · CineBrain test | reconstruction quality | Improved |
Closest papers by meaning
- SIM: Surface-based fMRI Analysis for Inter-Subject Multimodal Decoding from Movie-Watching Experiments (ICLR 2025 · similarity 0.89)
- SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning (NeurIPS 2025 · similarity 0.87)
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.87)
- EEG2Video: Towards Decoding Dynamic Visual Perception from EEG Signals (NeurIPS 2024 · similarity 0.87)
- Multi-modal brain encoding models for multi-modal stimuli (ICLR 2025 · similarity 0.87)
- Brain decoding: toward real-time reconstruction of visual perception (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
