Research map / Time series / Long-term forecasting
RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RhythmMamba, a state-space-model (Mamba) architecture for remote photoplethysmography that embeds spatial information into channels via a frame stem, adds multi-temporal constraint Mamba blocks and a frequency-domain feed-forward, achieving SOTA accuracy with much higher throughput and lower memory.
Problem
Remote rPPG measurement from facial video needs both long-range periodic modeling and handling of large spatiotemporal redundancy, under a compute budget suitable for mobile deployment; directly applying Mamba with patch embeddings performs poorly.
Approach
Treat rPPG as a time-series task by using a frame stem that folds spatial information into channels so that each channel is a purely temporal sequence for Mamba's state transitions; add multi-temporal constraint slicing of sequences with one shared Mamba block and a frequency-domain feed-forward that does channel interaction via FFT.
Key results
- On PURE intra-dataset, achieves MAE 0.23 and RMSE 0.34, best across all metrics reported.
- On MMPD intra-dataset, MAE 3.16 and RMSE 7.27 versus RhythmFormer's 3.07 and 6.81, so comparable rather than best on that dataset.
- Reports 319% throughput and 23% peak GPU memory relative to baselines with 1.07M parameters and 80.9M MACs per frame.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RhythmMamba instead of RhythmFormer | rPPG measurement · PURE intra-dataset | MAE: 0.27 -> 0.23 | Helped |
| Use RhythmMamba instead of NEST | rPPG measurement · VIPL-HR intra-dataset | MAE: 4.76 -> 4.30 | Helped |
| Use RhythmMamba instead of RhythmFormer | rPPG measurement · MMPD intra-dataset | MAE: 3.07 -> 3.16 | Hurt |
| Add diff-fusion (ablation) | RhythmMamba · MMPD | MAE: 5.71 -> 3.16 | Helped |
| Add self-attention (ablation) | RhythmMamba · MMPD | MAE: 4.02 -> 3.16 | Helped |
| Add large convolution kernels (ablation) | RhythmMamba · MMPD | MAE: 3.51 -> 3.16 | Helped |
| Add multi-temporal constraint (ablation) | RhythmMamba · MMPD | MAE: 3.60 -> 3.16 | Helped |
| Add frequency-domain FFN (ablation) | RhythmMamba · MMPD | MAE: 3.54 -> 3.16 | Helped |
| Replace frequency-domain FFN with Spatio-Temporal FFN (ablation) | RhythmMamba · MMPD | MAE: 3.16 -> 3.82 | Hurt |
| Represent spatial information as 1×1 average-pooled temporal tokens instead of 4×4 spatial tokens (ablation) | RhythmMamba with vanilla FFN · MMPD | MAE: 4.62 -> 3.54 | Helped |
| Use 1×1 average pooling instead of 8×8 spatial tokens (ablation) | RhythmMamba with vanilla FFN · MMPD | MAE: 4.90 -> 3.54 | Helped |
| Use 1×1 average pooling instead of 4×4 temporal embedding (ablation) | RhythmMamba with vanilla FFN · MMPD | MAE: 4.92 -> 3.54 | Helped |
Surprising findings
- Tokenized spatiotemporal input (patch embedding with spatial tokens) degrades Mamba's rPPG performance, with spatial information disrupting temporal modeling.
Limitations
- Improvements on intra-dataset MMPD and cross-dataset results are less significant than on PURE; the paper suggests fine-grained self-attention may help domain-invariant features.
- Utilization of periodic priors is currently limited, as stated in the conclusion.
- UBFC evaluation uses last-epoch checkpoints without a validation set.
Experimental setup
Datasets: PURE UBFC-rPPG VIPL-HR MMPD
Benchmarks: PURE UBFC-rPPG VIPL-HR MMPD
Metrics: MAE RMSE MAPE Pearson correlation SNR throughput peak GPU memory MACs parameters
Baselines: DeepPhys PhysNet TS-CAN PhysFormer EfficientPhys RhythmFormer Spiking-PhysFormer GREEN ICA CHROM POS LGI
Builds on: Mamba PhysNet PhysFormer RhythmFormer diff-fusion
Closest papers by meaning
- Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios (CVPR 2025 · similarity 0.90)
- PhysDiff: Physiology-based Dynamicity Disentangled Diffusion Model for Remote Physiological Measurement (AAAI 2025 · similarity 0.90)
- FacePhys: State of the Heart Learning (NeurIPS 2026 · similarity 0.89)
- FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing (NeurIPS 2024 · similarity 0.87)
- PaPaGei: Open Foundation Models for Optical Physiological Signals (ICLR 2025 · similarity 0.85)
- Fusion-Vital: Video-RF Fusion Transformer for Advanced Remote Physiological Measurement (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.