Research map / Label-efficient and robust learning / Self-supervised and contrastive
Large-scale Training of Foundation Models for Wearable Biosignals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pre-trains PPG and ECG foundation models with participant-level contrastive SSL (regularized InfoNCE + KoLeo, momentum encoder) on ~141K Apple Watch users and probes the embeddings for demographics and health conditions.
Problem
Wearable biosignal datasets with medical labels are small, limiting neural models for PPG and ECG; the paper asks whether large-scale consumer wearable data can yield useful foundation models.
Approach
Participant-level positive pairs (two different segments from the same person) with a stochastic augmentation cascade, a KoLeo-regularized InfoNCE loss, and momentum-updated encoder, applied to a very large consumer-wearable PPG/ECG corpus.
Key results
- PPG embeddings reached 0.976 AUC for age>50 classification and 0.993 AUC for sex with participant-level linear probing; ECG reached 0.916 and 0.951.
- PPG embeddings beat demographic and heart-rate baseline features on most of the health-condition targets (e.g., Pacemaker 0.897 vs 0.813 baseline, Antidepressants 0.837 vs 0.658).
- Participant-level positive pairs outperformed segment-level pairs on all downstream targets for both modalities; the full framework had higher smooth effective rank than SimCLR and BYOL variants (PPG 104.13 vs 98.87 and 51.18).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use participant-level rather than segment-level positive pairs (ablation) | PPG foundation model · Age classification | AUC: 0.900 -> 0.976 | Helped |
| use participant-level rather than segment-level positive pairs (ablation) | PPG foundation model · BMI classification | AUC: 0.766 -> 0.918 | Helped |
| use participant-level rather than segment-level positive pairs (ablation) | PPG foundation model · Sex classification | AUC: 0.870 -> 0.993 | Helped |
| use participant-level rather than segment-level positive pairs (ablation) | ECG foundation model · Age classification | AUC: 0.783 -> 0.916 | Helped |
| use participant-level rather than segment-level positive pairs (ablation) | ECG foundation model · BMI classification | AUC: 0.734 -> 0.797 | Helped |
| use participant-level rather than segment-level positive pairs (ablation) | ECG foundation model · Sex classification | AUC: 0.854 -> 0.951 | Helped |
| add KoLeo regularization to the objective (ablation) | PPG pre-training framework · PPG pre-training | Smooth effective rank: 101.29 -> 104.13 | Helped |
| add KoLeo regularization to the objective (ablation) | ECG pre-training framework · ECG pre-training | Smooth effective rank: 108.84 -> 113.82 | Helped |
| use the proposed framework instead of SimCLR | PPG pre-training · PPG pre-training | Smooth effective rank: 98.87 -> 104.13 | Helped |
| use the proposed framework instead of SimCLR | ECG pre-training · ECG pre-training | Smooth effective rank: 103.65 -> 113.82 | Helped |
| use the proposed framework instead of BYOL | PPG pre-training · PPG pre-training | Smooth effective rank: 51.18 -> 104.13 | Helped |
| use the proposed framework instead of BYOL | ECG pre-training · ECG pre-training | Smooth effective rank: 63.67 -> 113.82 | Helped |
Surprising findings
- ECG embeddings had lower InfoNCE loss and higher smooth effective rank than PPG yet were less predictive of demographics and health conditions, so unsupervised metrics do not track downstream performance monotonically.
- BYOL-style non-contrastive pre-training gave much lower smooth effective rank (51.18 PPG) than the contrastive variants.
Limitations
- Health labels come from opportunistic self-reported survey responses, which are sub-optimal.
- ECG embeddings are less predictive than PPG, and the reason is not resolved; the framework is not modality-specific.
- Data cannot be shared due to the IRB-approved consent language, limiting reproducibility.
Experimental setup
Datasets: Apple Heart and Movement Study PPG Apple Heart and Movement Study ECG AHMS survey questionnaires
Benchmarks: AHMS demographics (age, BMI, biological sex) AHMS self-reported health conditions and medications
Metrics: AUC pAUC at 10% FPR MAE smooth effective rank
Baselines: SimCLR (own variation) BYOL (own variation) 1D-ResNet 1D-ViT baseline demographic and heart rate features
Builds on: SimCLR BYOL InfoNCE EfficientNet
Closest papers by meaning
- Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions (ICML 2025 · similarity 0.88)
- OpenMHC: Accelerating the Science of Wearable Foundation Models (NeurIPS 2026 · similarity 0.88)
- Scaling Wearable Foundation Models (ICLR 2025 · similarity 0.87)
- PaPaGei: Open Foundation Models for Optical Physiological Signals (ICLR 2025 · similarity 0.86)
- WildPPG: A Real-World PPG Dataset of Long Continuous Recordings (NeurIPS 2024 · similarity 0.84)
- From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.