Research map / Vision-language and multimodal / Cross-modal learning
MindMix: A Multimodal Foundation Model for Auditory Perception Decoding via Deep Neural-Acoustic Alignment
RUI LIU, Zhige Chen, Pengshu, Wenlong You, Zhi-An Huang, Jibin Wu, KC Tan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CALRA cross-modal alignment instead of CLIP-style projection (ablation) | MindMix · KUL | balanced accuracy: 0.9535 -> 0.9982 | Improved |
| use CALRA instead of concat-MLP fusion (ablation) | MindMix · HR-EEG4EMO | balanced accuracy: 0.8574 -> 0.8878 | Improved |
| add bi-directional cross-attention in CALRA (ablation) | MindMix · KUL | balanced accuracy: 0.9435 -> 0.9982 | Improved |
| add shared low-rank alignment in CALRA (ablation) | MindMix · KUL | balanced accuracy: 0.9742 -> 0.9982 | Improved |
| add type-specific aligner in CALRA (ablation) | MindMix · KUL | balanced accuracy: 0.9853 -> 0.9982 | Improved |
| use the proposed pretrained EEG encoder instead of LaBraM/CBraMod/EEGNet (ablation) | MindMix · KUL | balanced accuracy: 0.9637 (CBraMod) -> 0.9982 | Improved |
| use Wav2Vec 2.0 instead of Mel-spectrogram audio features (ablation) | MindMix · KUL | balanced accuracy: 0.9448 -> 0.9982 | Improved |
| use Wav2Vec 2.0 instead of HuBERT (ablation) | MindMix · KUL | balanced accuracy: 0.9883 -> 0.9982 | Improved |
| add audio alignment (multimodal) to EEG encoder (ablation) | MindMix · DTU (between-trial) | balanced accuracy: 0.6557 -> 0.9543 | Improved |
| use trial-disjoint evaluation instead of within-subject random split | DBPNet/DARNet baselines · KUL | balanced accuracy: 0.9481 -> 0.6536 (DARNet) | Hurt |
| lengthen decision window from 2s to 5s (ablation) | MindMix · HR-EEG4EMO | balanced accuracy: 0.8878 -> 0.8917 | No clear effect |
| shorten decision window from 2s to 1s (ablation) | MindMix · HR-EEG4EMO | balanced accuracy: 0.8878 -> 0.8535 | Hurt |
Closest papers by meaning
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.87)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.87)
- BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.87)
- SENSE: Semantic Neural Speech Synthesis from Brain Dynamics via Spatial Graph Encoding (NeurIPS 2026 · similarity 0.87)
- Neural decoding from stereotactic EEG: accounting for electrode variability across subjects (NeurIPS 2024 · similarity 0.87)
- The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
