Research map / Vision-language and multimodal / Cross-modal learning
Robust Sleep Staging over Incomplete Multimodal Physiological Signals via Contrastive Imagination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CIMSleepNet, a multimodal sleep staging network robust to incomplete (chunk-wise missing) EEG/EOG/EMG signals, combining a variational modality-imagination module, semantic-and-modality calibrated contrastive loss, and multi-level cross-branch temporal attention.
Problem
Automated sleep staging from multimodal physiological signals degrades sharply when modalities are missing (sensor malfunction or detachment), and existing temporal models capture context at only one level (intra- or inter-epoch).
Approach
Impute missing modalities with a variational multimodal autoencoder (MAIM), then align the recovered data distribution with a three-level contrastive loss (SMCCL) whose weights depend on both label and modality agreement and on an inter-modal mutual-information score; temporal context is modeled with MCTA, a Transformer whose attention mixes temporal-CNN and Bi-GRU branches at intra- and inter-epo
Key results
- On Sleep-EDF-20 at maximum missing rate, CIMSleepNet reaches 0.853 accuracy / 0.801 MF1 / 0.805 kappa versus 0.623 accuracy for the best baseline XSleepNet.
- In the completely-missing setting (trained at rho=0.5), CIMSleepNet with EEG+EOG reaches 0.903 accuracy / 0.828 MF1 / 0.862 kappa, above CoRe-Sleep at 0.895 / 0.823 / 0.853.
- Ablation on Sleep-EDF-20 shows removing MAIM, SMCCL, or MCTA each lowers performance, with the full model reaching 0.853 accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CIMSleepNet instead of the strongest baseline | Sleep staging with incomplete modalities · Sleep-EDF-20 incomplete modalities | Acc: 0.634 -> 0.853 | Helped |
| Use CIMSleepNet instead of the strongest baseline | Sleep staging with incomplete modalities · Sleep-EDF-78 incomplete modalities | Acc: 0.738 -> 0.830 | Helped |
| Use CIMSleepNet instead of the strongest baseline | Sleep staging with incomplete modalities · SVUH-UCD incomplete modalities | Acc: 0.725 -> 0.788 | Helped |
| Use CIMSleepNet instead of the strongest baseline | Sleep staging with incomplete modalities · MHR incomplete modalities | Acc: 0.616 -> 0.701 | Helped |
| Use CIMSleepNet instead of the strongest baseline | Sleep staging with complete modalities · MHR complete modalities | Acc: 0.759 -> 0.729 | Hurt |
| Use CIMSleepNet instead of CoRe-Sleep | SHHS with completely missing modalities · SHHS EEG test modality | Acc: 0.882 -> 0.891 | Helped |
| Use CIMSleepNet instead of CoRe-Sleep | SHHS with completely missing modalities · SHHS EOG test modality | Acc: 0.853 -> 0.858 | Helped |
| Use CIMSleepNet instead of CoRe-Sleep | SHHS with completely missing modalities · SHHS EEG+EOG test modalities | Acc: 0.895 -> 0.903 | Helped |
| add MAIM (ablation) | CIMSleepNet · Sleep-EDF-20, missing rate ρ = 0.5 | Acc: 0.497 -> 0.771 | Helped |
| add SMCCL (ablation) | CIMSleepNet · Sleep-EDF-20, missing rate ρ = 0.5 | Acc: 0.497 -> 0.786 | Helped |
| add MCTA (ablation) | CIMSleepNet · Sleep-EDF-20, missing rate ρ = 0.5 | Acc: 0.497 -> 0.694 | Helped |
| add the KL divergence loss term (ablation) | CIMSleepNet MAIM · Sleep staging | performance | Helped |
Surprising findings
- Removing the KL term from the imagination loss causes a significant performance drop, while the KL weight eta is reported as insensitive.
- On complete modalities CIMSleepNet is only comparable to baselines (0.867 vs. 0.872 accuracy for SalientSleepNet on Sleep-EDF-20), so gains come mainly under missing data.
Limitations
- Additional architecture adds computational overhead (model size about 37.7 MB and 2.9 GFLOPs in the ablation).
- Requires labeled data; the authors note that label scarcity is common and that semi-supervised extension is future work.
- Privacy concerns with sensitive physiological data are acknowledged in broader impacts.
Experimental setup
Datasets: Sleep-EDF-20 Sleep-EDF-78 SVUH-UCD MHR SHHS-1
Benchmarks: Sleep-EDF-20 Sleep-EDF-78 SVUH-UCD MHR SHHS-1
Metrics: accuracy macro F1 Cohen's kappa
Baselines: FeatConcat MultitaskCNN SalientSleepNet MM-Net TransSleep XSleepNet MLP DeepCNN CoRe-Sleep ICL SCL
Builds on: MVAE CoRe-Sleep TransSleep XSleepNet supervised contrastive learning
Closest papers by meaning
- SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals (ICML 2024 · similarity 0.90)
- Subject-Relative Micro-Motion and Sleep Dynamics for Near-Infrared Video Sleep Staging (NeurIPS 2026 · similarity 0.85)
- Generalizable Sleep Staging via Multi-Level Domain Alignment (AAAI 2024 · similarity 0.85)
- Personalized Sleep Staging Leveraging Source-free Unsupervised Domain Adaptation (AAAI 2025 · similarity 0.84)
- CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning (NeurIPS 2025 · similarity 0.82)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.