Research map / Video / Video segmentation and tracking
Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ST-MEM, a masked autoencoder that pre-trains a ViT-B on 12-lead ECG using spatio-temporal patchifying and lead-indicating modules, showing gains in arrhythmia classification, low-resource, and reduced-lead settings.
Problem
Self-supervised representation learning for ECG ignores the spatial (lead) and temporal structure of multi-lead signals, limiting performance when labeled ECG data are scarce and when only reduced lead sets are available.
Approach
Masked ECG modeling with spatio-temporal patchifying (each lead split into temporal patches, all leads' patches fed to one encoder), a lead-wise shared decoder that prevents cross-lead copying, and learnable lead embeddings plus SEP tokens to indicate lead identity.
Key results
- On PTB-XL 12-lead linear evaluation, ST-MEM reaches AUROC 0.838 and F1 0.508, versus 0.807 (MTAE) and 0.797 (CMSC).
- With 1% of labels on CPSC2018, ST-MEM fine-tuning reaches AUROC 0.897 versus 0.860 for MLAE and 0.600 for supervised.
- On PhysioNet2017 single-lead (lead I), ST-MEM reaches AUROC 0.866 versus 0.861 for MLAE and 0.857 for MTAE+RLM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · PTB-XL, linear evaluation | AUROC: 0.807 ± 0.006 -> 0.838 ± 0.011 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · CPSC2018, linear evaluation | F1: 0.349 ± 0.034 -> 0.641 ± 0.010 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · PTB-XL, fine-tuning | AUROC: 0.915 ± 0.001 -> 0.933 ± 0.003 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · CPSC2018, fine-tuning | AUROC: 0.973 ± 0.002 -> 0.980 ± 0.001 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · PTB-XL, 1% labeled data | AUROC: 0.797 ± 0.006 -> 0.815 ± 0.012 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · PTB-XL, 5% labeled data | AUROC: 0.838 ± 0.018 -> 0.878 ± 0.011 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · CPSC2018, 1% labeled data | AUROC: 0.860 ± 0.013 -> 0.897 ± 0.025 | Helped |
| Use ST-MEM instead of the strongest listed baseline | ViT-B ECG representation · CPSC2018, 5% labeled data | AUROC: 0.922 ± 0.007 -> 0.952 ± 0.004 | Helped |
| Add a lead-wise shared decoder (ablation) | ST-MEM · PTB-XL, linear evaluation | AUROC: 0.805 -> 0.822 | Helped |
| Add a lead-wise shared decoder (ablation) | ST-MEM · CPSC2018, linear evaluation | AUROC: 0.861 -> 0.911 | Helped |
| Add SEP embedding after the lead-wise shared decoder (ablation) | ST-MEM · PTB-XL, linear evaluation | AUROC: 0.822 -> 0.820 | Hurt |
| Add SEP embedding after the lead-wise shared decoder (ablation) | ST-MEM · CPSC2018, linear evaluation | AUROC: 0.911 -> 0.929 | Helped |
Surprising findings
- MoCo v3 linear evaluation on PTB-XL is near chance-level in F1 (0.142), and CMSC fine-tuning falls below supervised learning.
- Flip, shift, and sine augmentations are argued to distort ECG semantics, so the MoCo v3 baseline uses only four of eight augmentations for some comparisons.
Limitations
- Downstream datasets are relatively small and label sets are limited to arrhythmia and MI tasks.
- Pre-training data is restricted to three public 12-lead ECG datasets, and results are reported with mean and std over few runs.
- Contrastive baselines are reproduced by the authors, so performance may differ from original implementations.
Experimental setup
Datasets: Chapman Ningbo CODE-15 PTB-XL CPSC2018 PhysioNet2017
Benchmarks: PTB-XL CPSC2018 PhysioNet2017
Metrics: accuracy F1 AUROC
Baselines: Supervised MoCo v3 CMSC MTAE MTAE+RLM MLAE CPC
Builds on: Masked Autoencoder (MAE) Vision Transformer (ViT) MaeFE MoCo v3 CLOCS
Closest papers by meaning
- Parallel-Learning of Invariant and Tempo-variant Attributes of Single-Lead Cardiac Signals: PLITA (AAAI 2025 · similarity 0.88)
- From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.86)
- Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model (ICLR 2025 · similarity 0.85)
- Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners (ICML 2025 · similarity 0.84)
- Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement (ICML 2024 · similarity 0.83)
- GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.