Research map / Time series / Foundation models for time series
Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Treats heartbeats as words and rhythms as sentences, using a QRS-based tokenizer, a VQ heartbeat vocabulary of 8,192 codes, and masked ECG-sentence pre-training (HeartLang) for self-supervised ECG representation learning.
Problem
Existing ECG self-supervised learning methods treat ECG as ordinary time series with fixed-size windows, ignoring heartbeat morphology, rhythm structure, and latent semantic relations between heartbeats.
Approach
Segment ECG at QRS complexes into heartbeat "words" and assemble them into rhythm "sentences"; learn a discrete heartbeat vocabulary via vector-quantized reconstruction, then pre-train a transformer by masking heartbeats and predicting their vocabulary indices.
Key results
- Linear probing on PTB-XL with 100% data: HeartLang reaches 87.52 macro AUC on Superclass and 90.34 on Rhythm, the best among eSSL baselines in the table.
- On the CSN dataset with 100% training data, HeartLang achieves 82.49 macro AUC, the top score in the table, while it trails on CPSC2018 (77.87 vs 82.03 for CRT at 100%).
- Replacing the 8,192-word vocabulary with a 64-word vocabulary lowers PTB-XL Superclass 100% AUC from 87.52 to 86.23.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HeartLang instead of the strongest listed eSSL baseline | HeartLang · PTB-XL Superclass, Subclass, Form, and Rhythm (1% and 100% training data) | macro AUC: average macro AUC improvement of 8.14 | Helped |
| Use HeartLang instead of the strongest listed eSSL baseline | HeartLang · CPSC2018 | macro AUC | Hurt |
| Use HeartLang instead of the strongest listed eSSL baseline | HeartLang · CSN 100% training data | macro AUC | Helped |
| Replace fixed-size, fixed-step windows with heartbeat-as-word QRS segmentation (ablation) | HeartLang · PTB-XL Superclass, Subclass, Form, and Rhythm | macro AUC: 12.52; 7.71; 7.01; 3.81; 8.62; 10.58; -1.24; -1.53; 4.20; 6.26; 2.14; 4.29 | Mixed |
| Add ECG vocabulary instead of reconstructing masked segments with MSE (ablation) | HeartLang · PTB-XL Superclass, Subclass, Form, and Rhythm | macro AUC | Helped |
| Add masked ECG sentence pre-training (ablation) | HeartLang · PTB-XL downstream tasks | macro AUC | Helped |
| Add spatio-temporal embeddings (ablation) | HeartLang · PTB-XL downstream tasks | macro AUC | Helped |
| Add spatial embedding (ablation) | HeartLang · PTB-XL downstream tasks | macro AUC | Helped |
| Add temporal embedding (ablation) | HeartLang · PTB-XL downstream tasks | macro AUC | Helped |
| Increase ECG vocabulary size from 64 to 8192 (ablation) | HeartLang · PTB-XL Superclass, Subclass, Form, and Rhythm | macro AUC: 3.05; 1.87; 1.29; 3.71; 3.39; 2.03; 1.60; 1.00; 4.54; 3.67; 0.91; 1.83 | Helped |
| Use 12 leads instead of 1 lead (ablation) | HeartLang · PTB-XL Superclass, Subclass, Form, and Rhythm | macro AUC | Mixed |
Surprising findings
- The fixed-window slicing baseline achieves higher masked prediction accuracy during pre-training, yet HeartLang's QRS-based slicing yields better downstream performance.
Limitations
- QRS-Tokenizer may struggle with irregular QRS complexes in certain diseases.
- Zero-padding can disrupt heartbeat characteristics for signals with significant baseline drift, hurting CPSC2018 and CSN performance.
- Does not directly compare with multimodal MERL, only with other eSSL methods.
Experimental setup
Datasets: MIMIC-IV-ECG PTB-XL CPSC2018 Chapman-Shaoxing-Ningbo
Benchmarks: PTB-XL CPSC2018 Chapman-Shaoxing-Ningbo
Metrics: macro AUC
Baselines: SimCLR BYOL BarlowTwins MoCo-v3 SimSiam TS-TCC CLOCS ASTCL CRT ST-MEM
Builds on: VQ-NSP MERL BERT-style masked prediction
Closest papers by meaning
- From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.92)
- GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.87)
- Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement (ICML 2024 · similarity 0.87)
- Parallel-Learning of Invariant and Tempo-variant Attributes of Single-Lead Cardiac Signals: PLITA (AAAI 2025 · similarity 0.86)
- anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task Understanding (AAAI 2026 · similarity 0.86)
- ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.