Research map / Label-efficient and robust learning / Self-supervised and contrastive
Bootstrapping Vision-Language Models for Frequency-Centric Self-Supervised Remote Physiological Measurement
Zijie Yue, Miaojing Shi, Hanli Wang, Shuai Ding, Qijun Chen, Shanlin Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text modality (vision-text pairs, TVR, L_vtc, L_r) (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.57 -> 0.28 | Improved |
| add text modality (ablation) | VL-phys · MMVS (intra-dataset HR) | MAE: 2.88 -> 2.33 | Improved |
| add text-guided visual reconstruction (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.41 -> 0.28 | Improved |
| add vision-text contrastive loss (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.50 -> 0.28 | Improved |
| add frequency contrastive loss (ablation) | VL-phys · MMVS (intra-dataset HR) | MAE: 4.75 -> 2.33 | Improved |
| add frequency ranking loss (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.51 -> 0.28 | Improved |
| use relative rank loss instead of absolute ratio loss (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.33 -> 0.28 | Improved |
| use VindLU pre-trained weights instead of random init (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.55 -> 0.28 | Improved |
| use BERT text encoder instead of MLP on ratio token (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 1.57 -> 0.28 | Improved |
| swap VindLU for CLIP encoders (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.28 -> 0.49 | Hurt |
| replace full fine-tuning with VPT, CoOp or adapters (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.28 -> 0.61 | Hurt |
| use alternative text templates T1-T3 (ablation) | VL-phys · UBFC-rPPG (intra-dataset HR) | MAE: 0.28 -> 0.27 | No clear effect |
Closest papers by meaning
- Neuron Structure Modeling for Generalizable Remote Physiological Measurement (CVPR 2023 · similarity 0.83)
- Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling (ECCV 2020 · similarity 0.83)
- SensorLM: Learning the Language of Wearable Sensors (NeurIPS 2025 · similarity 0.80)
- RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement (AAAI 2025 · similarity 0.80)
- Fusion-Vital: Video-RF Fusion Transformer for Advanced Remote Physiological Measurement (AAAI 2024 · similarity 0.79)
- Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition - And Ways to Overcome Them (AAAI 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
