Research map / Video / Video segmentation and tracking
Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a lightweight U-shaped video transformer for rPPG that disentangles time-varying lighting interference using a background reference, self-supervised contrastive learning, and frequency/physiological priors, evaluated on RGB outdoor and indoor datasets.
Problem
Remote photoplethysmography (rPPG) from facial video fails under extreme, time-varying illumination and periodic interference in real-world (outdoor, driving) scenarios.
Approach
Uses the subject background as a reference for interference in a self-supervised contrastive disentanglement over facial and background STMaps, combined with a global-interference-sharing input, a biological-prior sliding-window augmentation (BioSE), a U-shaped Swin video transformer for spatiotemporal reconstruction, and PSD-based frequency constraints, all using RGB only.
Key results
- On MR-NIRP-DRV (trained on MR-NIRP), the model reaches RMSE 7.09 bpm, versus 7.49 for RhythmFormer and 7.52 for ND-DeeprPPG.
- On VIPL-HR, RMSE is 7.23 bpm with MAE 6.03 and correlation 0.87, the best among the compared methods.
- In RGB-only outdoor driving evaluation (Table 2), RMSE 7.30 day and 7.51 garage versus NIR-based methods, which the paper says are the strongest outdoor baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RGB instead of NIR imaging | Ours · MR-NIRP-DRV, Day | RMSE: 10.93 -> 5.45 | Helped |
| Use RGB instead of NIR imaging | Ours · MR-NIRP-DRV, Night | RMSE: 7.51 -> 3.84 | Hurt |
| Add interference disentanglement (ablation) | Ours · MR-NIRP-DRV | RMSE: 8.12 -> 6.03 | Helped |
| Use temporal sampling 80 rather than the full model setting (ablation) | Ours · MR-NIRP-DRV | RMSE: 6.03 -> 7.16 | Hurt |
| Use temporal sampling 160 rather than the full model setting (ablation) | Ours · MR-NIRP-DRV | RMSE: 6.03 -> 6.80 | Hurt |
| Use temporal sampling 640 rather than the full model setting (ablation) | Ours · MR-NIRP-DRV | RMSE: 6.03 -> 7.46 | Hurt |
| Use temporal frequency 2× rather than the full model setting (ablation) | Ours · MR-NIRP-DRV | RMSE: 6.03 -> 6.68 | Hurt |
| Use temporal frequency 4× rather than the full model setting (ablation) | Ours · MR-NIRP-DRV | RMSE: 6.03 -> 8.09 | Hurt |
| Use temporal frequency 8× rather than the full model setting (ablation) | Ours · MR-NIRP-DRV | RMSE: 6.03 -> 9.12 | Hurt |
Surprising findings
- Near-infrared based methods do not outperform RGB ones on the outdoor driving dataset, suggesting visible-light rPPG should be the main focus.
- The CNN variant of the disentanglement design is also evaluated to show gains come from architecture design, not only transformer self-attention.
Limitations
- Relies on public datasets with a limited set of scenarios; the in-the-wild driving case is still the hardest.
- Performance drops with longer temporal windows in some settings (Table 4), and the disentanglement design needs a background region outside the face.
Experimental setup
Datasets: BUAA-MIHR VIPL-HR MR-NIRP-IND MR-NIRP-DRV
Benchmarks: MR-NIRP-IND MR-NIRP-DRV VIPL-HR BUAA-MIHR
Metrics: MAE RMSE Pearson correlation
Baselines: CHROM POS LGI DeepPhys TS-CAN Dual-GAN PFE-TFA NEST ND-DeeprPPG EfficientPhys PhysFormer++ RhythmFormer
Builds on: Swin video transformer ND-DeeprPPG DeeprPPG PhysFormer++ RhythmFormer
Closest papers by meaning
- PhysDiff: Physiology-based Dynamicity Disentangled Diffusion Model for Remote Physiological Measurement (AAAI 2025 · similarity 0.90)
- RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement (AAAI 2025 · similarity 0.90)
- FacePhys: State of the Heart Learning (NeurIPS 2026 · similarity 0.88)
- FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing (NeurIPS 2024 · similarity 0.86)
- PaPaGei: Open Foundation Models for Optical Physiological Signals (ICLR 2025 · similarity 0.85)
- WildPPG: A Real-World PPG Dataset of Long Continuous Recordings (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.