Research map / Vision-language and multimodal / Human and embodied modeling
PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PhysDrive, a multimodal in-vehicle remote physiological measurement dataset (48 drivers, RGB, NIR, and mmWave radar with six contact ground truths) and benchmarks signal-processing and deep-learning rPPG/mmWave methods under real driving conditions.
Problem
Remote physiological measurement in real driving scenarios is hampered by a lack of comprehensive multimodal datasets covering realistic in-vehicle challenges such as lighting, driver motion, road conditions, and vehicle types.
Approach
A real-world driving data collection with synchronized RGB, NIR, and raw mmWave radar sensing paired with ECG, BVP, respiration, HR, RR and SpO2 ground truths, organized by lighting, vehicle type, driver action, and road condition factors, plus benchmarks across modalities.
Key results
- mmFormer achieves HR MAE 3.65 and Pearson 0.97 on PhysDrive intra-dataset, outperforming the best RGB method PhysNet (MAE 6.29, P 0.61).
- RGB and NIR methods degrade under bumpy congested roads; mmWave MAE increases about 13% compared with flat unobstructed roads.
- Traditional RGB methods such as CHROM and POS perform near chance on PhysDrive (P around 0.10), and models trained on PURE/UBFC-rPPG generalize poorly to PhysDrive scenarios.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mmWave rather than RGB sensing for HR estimation | PhysDrive intra-dataset HR baselines · PhysDrive intra-dataset | MAE: 6.29 -> 3.65 | Helped |
| Use mmWave rather than NIR sensing for HR estimation | PhysDrive intra-dataset HR baselines · PhysDrive intra-dataset | MAE: 10.69 -> 3.65 | Helped |
| Use STMap-based MTL rather than direct face-video MTL | RGB multi-task estimation · PhysDrive intra-dataset | RR MAE: 3.01 -> 2.70 | Helped |
| Remove Wave Recovery (ablation) | IQ-MVED · PhysDrive intra-dataset mmWave HR estimation | MAE: 14.17 -> 29.18 | Hurt |
| Remove Wave Recovery (ablation) | VitaNet · PhysDrive intra-dataset mmWave HR estimation | MAE: 4.94 -> 35.21 | Hurt |
| Remove Wave Recovery (ablation) | mmFormer · PhysDrive intra-dataset mmWave HR estimation | MAE: 3.65 -> 33.87 | Hurt |
| Drive while talking rather than stationary | PhysDrive HR and RR monitoring methods · PhysDrive cross-scenario evaluation | MAE | Hurt |
| Drive on bumpy and congested roads rather than flat, unobstructed roads | mmWave monitoring methods · PhysDrive cross-scenario evaluation | MAE: + about 13% | Hurt |
| Train RGB methods on PURE or UBFC-rPPG rather than PhysDrive | RGB HR methods · PhysDrive cross-dataset evaluation | MAE: 7.21 -> 9.30 | Hurt |
| Use RGB rather than NIR sensing | Vision-based HR methods · PhysDrive intra-dataset | MAE: 10.69 -> 6.29 | Helped |
Surprising findings
- mmWave methods outperform all vision-based methods on HR estimation, but struggle to reconstruct ECG waveforms.
- NIR methods do not exceed RGB performance despite being more stable under low light.
Limitations
- Participant cohort predominantly East Asian, limiting assessment of skin-tone bias in camera-based methods.
- Only drivers studied; passenger monitoring not covered.
- SpO2 variability is limited because no hypoxia induction was performed; synchronization tolerance around one second.
Experimental setup
Datasets: PhysDrive PURE UBFC-rPPG
Benchmarks: PhysDrive PURE UBFC-rPPG
Metrics: MAE RMSE Pearson correlation
Baselines: CHROM POS GREEN ICA ARM-RR Contrast-Phys+ SiNC DeepPhys PhysNet PhysFormer EfficientPhys RhythmFormer
Builds on: rPPG-Toolbox iBVP STMap
Closest papers by meaning
- MMTL-UniAD: A Unified Framework for Multimodal and Multi-Task Learning in Assistive Driving Perception (CVPR 2025 · similarity 0.83)
- DriveGazen: Event-Based Driving Status Recognition Using Conventional Camera (AAAI 2025 · similarity 0.82)
- Leveraging Driver Field-of-View for Multimodal Ego-Trajectory Prediction (ICLR 2025 · similarity 0.82)
- Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions (ICML 2025 · similarity 0.82)
- PhysioWave: A Multi-Scale Wavelet-Transformer for Physiological Signal Representation (NeurIPS 2025 · similarity 0.81)
- RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement (AAAI 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.