Research map / Video / Video segmentation and tracking
Subject-Relative Micro-Motion and Sleep Dynamics for Near-Infrared Video Sleep Staging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ViNUSS, a video-only NIR sleep staging framework combining pre-spatial micro-motion encoding (PTE), within-subject contrastive stage learning (ISDL), and two-scale Mamba-based full-night dynamics (SDM), reaching 0.80 accuracy and 0.78 MF1 on 475 KVSS recordings.
Problem
Video-based sleep staging from near-infrared video usually routes through reconstructed respiratory/cardiac proxies or EEG-guided cross-modal features; the paper asks whether NIR video alone carries informative PSG-defined sleep-stage evidence.
Approach
Encode dense pixel-wise temporal traces before spatial mixing (PTE), contrast epochs only within the same subject's night (ISDL), and model within-epoch motion with ShortMamba and full-night trajectories with bidirectional LongMamba, all from raw video without physiological proxies.
Key results
- ViNUSS reaches 0.80 accuracy, 0.68 kappa and 0.78 MF1 for four-class staging on KVSS test (70 subjects), versus 0.60 accuracy and 0.50 MF1 for vanilla MoViNet-A2.
- Full-oracle SleepVST (contact-sensor respiratory and cardiac waveforms) reaches 0.78 accuracy and 0.73 MF1 on KVSS, which ViNUSS exceeds in MF1 by 0.05.
- Adding SDM to the vanilla backbone raises MF1 from 0.50 to 0.67; adding PTE and ISDL on top yields 0.78 MF1.
Surprising findings
- Scaling MoViNet-A2 to MoViNet-A5 did not improve accuracy, and Video Swin-Tiny performed worse than the MoViNet variants.
- Video-only backbone variants without the sleep dynamics module reach only about 0.50-0.55 MF1 on KVSS.
Limitations
- Evaluated on a single hospital cohort (Hospital A) with no external validation across sites or home settings.
- Oracle SleepVST variants used contact-sensor signals, so the baseline comparison is not a fully video-only proxy-first method.
- Interpretability via SVECAM is qualitative and based on representative examples.
Experimental setup
Datasets: KVSS SHHS MESA
Benchmarks: KVSS
Metrics: accuracy Cohen's kappa macro-F1
Baselines: SleepVST MoViNet-A2 MoViNet-A5 Video Swin-Tiny van Meulen et al. Video-PSG BiLSTM PatchTST ModernTCN Vanilla Transformer
Builds on: MoViNet Mamba SleepVST EigenCAM
Closest papers by meaning
- SleepVST: Sleep Staging from Near-Infrared Video Signals using Pre-Trained Transformers (CVPR 2024 · similarity 0.89)
- SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals (ICML 2024 · similarity 0.86)
- Robust Sleep Staging over Incomplete Multimodal Physiological Signals via Contrastive Imagination (NeurIPS 2024 · similarity 0.85)
- Personalized Sleep Staging Leveraging Source-free Unsupervised Domain Adaptation (AAAI 2025 · similarity 0.84)
- Generalizable Sleep Staging via Multi-Level Domain Alignment (AAAI 2024 · similarity 0.83)
- PhysDiff: Physiology-based Dynamicity Disentangled Diffusion Model for Remote Physiological Measurement (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.