Research map / Vision-language and multimodal / Cross-modal learning
Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D-BETA, a masked ECG-text auto-encoder pre-trained with MLM, MEM, ETM plus a sigmoid-based ETS contrastive loss and nearest-neighbor negative sampling, evaluated on ECG classification, linear probing and zero-shot tasks.
Problem
Learning joint ECG-text representations is hard due to modality disparity and scarce labeled data, and existing ECG-text methods mostly use contrastive learning with standard encoders and poorly chosen negatives, leaving generative masked modeling underexplored.
Approach
Combine a multi-modal masked auto-encoder (masked language and ECG modeling with cross-attention fusion) with a sigmoid pairwise contrastive loss (ETS) on separate projection heads, and select hard negatives via nearest neighbors in Flan-T5 embedding space (N3S).
Key results
- Linear probing with 1% of PTB-XL-Rhythm data reaches 86.61 AUC versus MERL's 53.33, and CSN 1% AUC is improved by about 14 points over MERL.
- Zero-shot average AUC across six datasets is 77.1 versus MERL's 75.3 (about 2% gain), and zero-shot outperforms three cardiology residents/students in an expert comparison (96.79 vs 93.61 best human).
- Full fine-tuning on PhysioNet 2021 Dx with 12 leads reaches 85.7 CinC vs 73.2 for the best baseline W2V+CMSC+RLM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use D-BETA rather than W2V+CMSC+RLM | PhysioNet 2021 diagnosis classification, 12-lead · PhysioNet 2021 Dx. | CinC score: 73.2 -> 85.7 | Helped |
| Use D-BETA rather than W2V+CMSC+RLM | PhysioNet 2021 patient identification, 12-lead · PhysioNet 2021 Id. | accuracy: 57.7 -> 65.4 | Helped |
| Use D-BETA rather than MERL | zero-shot classification · PTBXL-Super, PTBXL-Sub, PTBXL-Form, PTBXL-Rhythm, CSN | AUC: +2.0, +0.2, +0.2, +10.1, +1.9 | Helped |
| Use D-BETA rather than MERL | zero-shot classification · CPSC2018 | AUC: 82.8 -> 80.1 | Hurt |
| Add N3S negative sampling rather than randomly sampling negatives (ablation) | D-BETA · Ablation tasks | score: 80.93 ± 3.74 vs 81.88 ± 3.52; 78.29 ± 6.19 vs 80.52 ± 6.08; 70.61 ± 8.10 vs 72.50 ± 9.01 | Helped |
| Add ETS loss (ablation) | D-BETA · Ablation tasks | score: 76.81 ± 3.96 -> 78.29 ± 3.87; 63.50 ± 6.95 -> 67.19 ± 6.14 | Helped |
| Use Flan-T5 instead of BERT-base (ablation) | D-BETA text encoder · Linear probing | score: 63.50 ± 6.95 -> 80.52 ± 6.08 | Helped |
| Increase ECG transformer layers from 1 to 8 (ablation) | D-BETA ECG encoder · Full fine-tuning | score: 69.40 ± 4.55 -> 81.88 ± 3.52 | Helped |
| Increase ECG transformer layers from 1 to 8 (ablation) | D-BETA ECG encoder · Linear probing | score: 66.83 ± 7.52 -> 80.52 ± 6.08 | Helped |
| Use 8 rather than 4 ECG transformer layers (ablation) | D-BETA ECG encoder · Zero-shot classification | score: 70.64 ± 8.63 -> 72.50 ± 9.01 | Helped |
| Use Flan-T5 rather than BERT (ablation) | D-BETA text encoder · Zero-shot classification | score: 69.14 ± 9.97 -> 72.50 ± 9.01 | Helped |
| Add ETM loss (ablation) | D-BETA · Zero-shot PTBXL-Super | AUC: 73.2 -> 76.2 | Helped |
Surprising findings
- Removing the ETS loss causes roughly 15% drops in both full fine-tuning and linear probing, indicating the discriminative loss matters more than the masked modeling alone in the ablation.
- The paper reports that a 1-layer ECG encoder still beats MERL in zero-shot by about 8% without GPT-4o text enhancement.
Limitations
- Zero-shot results rely on GPT-4o-generated category descriptions, and the ablations without it show lower performance.
- Some reported ablation tables are inconsistent with the main results (e.g., the ETM and zero-shot ablations only on four datasets), and the paper reports single-run standard deviations without clear significance testing.
- Zero-shot gain over MERL is small on CPSC2018, where MERL performs better.
Experimental setup
Datasets: MIMIC-IV-ECG PhysioNet 2021 PTB-XL CSN CPSC2018 CODE-test
Benchmarks: PhysioNet 2021 PTB-XL CSN CPSC2018 CODE-test
Metrics: AUC CinC score accuracy
Baselines: SimCLR BYOL BarlowTwins MoCo-v3 SimSiam TS-TCC CLOCS ASTCL CRT ST-MEM MERL W2V+CMSC+RLM
Builds on: MERL SigLIP ST-MEM CLIP Flan-T5
Closest papers by meaning
- From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.86)
- GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.86)
- Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement (ICML 2024 · similarity 0.85)
- Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model (ICLR 2025 · similarity 0.85)
- Robustness and Visual Explanation for Black Box Image, Video, and ECG Signal Classification with Reinforcement Learning (AAAI 2024 · similarity 0.85)
- ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.