Research map / Large language models / Language, safety and interpretability
Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MERL, a multimodal ECG-report contrastive pretraining framework with latent dropout augmentation, plus CKEPE, an LLM-driven test-time prompt engineering method grounded in clinical knowledge databases, enabling zero-shot ECG classification and a six-dataset benchmark.
Problem
Self-supervised ECG representation learning ignores clinical knowledge in reports, requires annotated data for downstream tasks, and cannot perform zero-shot classification; naive signal augmentations also distort ECG semantics.
Approach
Align ECG records with their clinical reports via cross-modal contrastive learning, replace input-level augmentation with latent dropout for uni-modal contrastive learning, and at test time use GPT-4 with web browsing and local/SNOMED clinical databases to generate structured, verified descriptive prompts for zero-shot classification.
Key results
- MERL zero-shot averages 75.2% AUC across six datasets without training data, 3.2% higher than linear-probed eSSL with 10% labeled data.
- Linear-probed MERL with 1% data on PTBXL-Super (82.39) exceeds all eSSL methods with 100% data on that dataset.
- Ablation: Med-CPT text encoder gives 75.24 zero-shot vs 72.36 for BioClinicalBERT; latent dropout (0.1) beats cutout, drop, and Gaussian noise augmentations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add UMA to CMA training (ablation) | MERL · Six ECG datasets, averaged | zero-shot AUC: 60.84±3.8 -> 61.67±4.2 | Helped |
| Add UMA to CMA training (ablation) | MERL · Six ECG datasets, 1% linear probing average | AUC: 64.25±2.6 -> 65.96±2.1 | Helped |
| Replace Med-CPT with BioClinicalBERT (ablation) | MERL · Six ECG datasets, averaged | zero-shot AUC: 75.24±1.7 -> 72.36±4.5 | Hurt |
| Replace Med-CPT with BioClinicalBERT (ablation) | MERL · Six ECG datasets, 1% linear probing average | AUC: 65.96±2.1 -> 63.81±1.8 | Hurt |
| Replace Med-CPT with PubMedBERT (ablation) | MERL · Six ECG datasets, averaged | zero-shot AUC: 75.24±1.7 -> 71.84±3.2 | Hurt |
| Replace Med-CPT with PubMedBERT (ablation) | MERL · Six ECG datasets, 1% linear probing average | AUC: 65.96±2.1 -> 64.29±2.5 | Hurt |
| Remove SNOMEDCT web database (ablation) | CKEPE · Six ECG datasets, averaged | zero-shot AUC: 75.24±1.7 -> 72.17±2.3 | Hurt |
| Remove SCP Statement local database (ablation) | CKEPE · Six ECG datasets, averaged | zero-shot AUC: 75.24±1.7 -> 73.62±1.9 | Hurt |
| Replace latent dropout with raw-signal cutout (ablation) | MERL · Six ECG datasets, averaged | zero-shot AUC: 75.24±1.7 -> 73.24±3.2 | Hurt |
| Replace latent dropout with raw-signal cutout (ablation) | MERL · Six ECG datasets, 1% linear probing average | AUC: 65.96±2.1 -> 62.24±2.7 | Hurt |
| Replace latent dropout with raw-signal drop (ablation) | MERL · Six ECG datasets, averaged | zero-shot AUC: 75.24±1.7 -> 72.79±2.5 | Hurt |
| Replace latent dropout with raw-signal drop (ablation) | MERL · Six ECG datasets, 1% linear probing average | AUC: 65.96±2.1 -> 61.14±2.2 | Hurt |
Surprising findings
- A randomly initialized ResNet18 achieves the second-highest linear-probing result on several PTBXL subsets with 100% data, ahead of all eSSL methods.
- Removing the web-based SNOMEDCT database hurts zero-shot performance more than removing the local SCP statement database.
Limitations
- CKEPE relies on GPT-4 generation whose process is not fully controllable or transparent, per the authors' broader impact statement.
- Zero-shot MERL underperforms ST-MEM on the PTBXL-Super to CSN distribution shift setting.
- The local SCP statement database is not publicly released, and the CPSC2018/CSN splits are to be released only after acceptance.
Experimental setup
Datasets: MIMIC-ECG PTBXL CPSC2018 Chapman-Shaoxing-Ningbo (CSN) SNOMEDCT SCP statements
Benchmarks: PTBXL-Super PTBXL-Sub PTBXL-Form PTBXL-Rhythm CPSC2018 CSN
Metrics: AUC (macro)
Baselines: SimCLR BYOL BarlowTwins MoCo-v3 SimSiam TS-TCC CLOCS ASTCL CRT ST-MEM
Builds on: CLIP SimCLR Med-CPT ST-MEM
Closest papers by meaning
- From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.88)
- Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model (ICLR 2025 · similarity 0.87)
- GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.86)
- anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task Understanding (AAAI 2026 · similarity 0.86)
- Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners (ICML 2025 · similarity 0.85)
- ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.