Research map / Video / Video segmentation and tracking
FacePhys: State of the Heart Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FacePhys, a memory-efficient rPPG model built on a neural-CDE-inspired heart-state SSM with temporal-spatial state space duality, enabling single-frame real-time inference and training on long videos.
Problem
Remote photoplethysmography (rPPG) deep models are too computationally heavy and short-clip-limited for real-time on-device deployment, while accuracy and cross-dataset generalization also need to hold.
Approach
Model the blood volume pulse as a discretized linear controlled differential equation solved with a state space model (SSD dual), using complex diagonal state transition entries for periodic attention, temporal normalization with recursive moving average for streaming, and 2D spatial convs with temporal SSM so inference runs frame by frame.
Key results
- Intra-dataset MAE on MMPD of 5.58 vs 8.13 for the best prior baseline (PhysNet), with cross-dataset MMPD MAE of 5.30 when trained on RLAP.
- Per-frame CPU latency of 9.46 ms and memory footprint of 3.6 MB with 719K parameters, versus 889 ms latency for RhyMamba.
- Reported up to 49% overall error reduction and 83% to 99% latency improvement over existing methods.
Surprising findings
- Removing the oscillator (complex-valued A) gives a lower MAE on VitalVideo and UBFC than the full model in some cases, and RhyMamba beats FacePhys on PURE and UBFC intra-dataset Pearson correlation ties.
Limitations
- Single-frame causal inference introduces lag since future frames are not accessible.
- Not tested in clinical settings with cardiovascular patients.
- Inconsistent reporting: abstract and conclusion claim different improvement figures (49%, 60.3%), and the paper's own gain tables use subset comparisons.
Experimental setup
Datasets: UBFC-rPPG PURE MMPD RLAP-rPPG VitalVideo
Benchmarks: MMPD VitalVideo PURE UBFC
Metrics: MAE RMSE Pearson correlation latency memory footprint FLOPs
Baselines: EFFPhys TSCAN PhysNet PhysFormer RhythmMamba
Builds on: Mamba-2 S4D Neural CDE PhysNet Temporal Normalization
Closest papers by meaning
- RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement (AAAI 2025 · similarity 0.89)
- Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios (CVPR 2025 · similarity 0.88)
- PhysDiff: Physiology-based Dynamicity Disentangled Diffusion Model for Remote Physiological Measurement (AAAI 2025 · similarity 0.88)
- FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing (NeurIPS 2024 · similarity 0.87)
- PaPaGei: Open Foundation Models for Optical Physiological Signals (ICLR 2025 · similarity 0.86)
- Fusion-Vital: Video-RF Fusion Transformer for Advanced Remote Physiological Measurement (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.