Research map / Vision-language and multimodal / Multimodal LLMs

anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task Understanding

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds the anyECG dataset (report generation, long/reduced-lead waveform localization, multi-ECG QA) and anyECG-chat, a Llama-3-8B-based ECG MLLM with dynamic-length and multi-ECG input, trained with a three-stage curriculum.

Problem

Existing ECG multimodal LLMs handle mainly single 12-lead 10-second ECGs and report generation, so they cannot do fine-grained localization, long or reduced-lead recordings, or multi-ECG comparison.

Approach

A ViT-based ECG encoder with lead embeddings and spatio-temporal patching, a dynamic ECG input scheme (zero-padding, 10-second segmentation with averaged CLS, ECG start/end tokens for multiple ECGs), and a three-stage curriculum (report generation, then localization with LoRA, then instruction and multi-ECG QA) on a new anyECG dataset.

Key results

• On six OOD ReportGen datasets, anyECG-chat reaches macro-AUC 68.95 on PTBXL-Super and 71.05 on CPSC, beating generative zero-shot PULSE and MEIT, though discriminative MERL is competitive on several sets.
• On MIMIC Multi-ECG QA, anyECG-chat averages 3.02 versus 1.53 for PULSE, 1.90 for MEIT and 2.48 for LLaVa-Med, and is judged best on 816 of 1,152 questions.
• On ECG-QA, anyECG-chat (10% training data) gets 70.1% on CI-Verify and 67.9% on CC-Verify, the best among generative ECG-MLLMs but below discriminative baselines.

Surprising findings

• Ablation shows removing experience replay leaves Multi-ECG score unchanged (3.04) while Classification drops sharply, and full-parameter tuning gives essentially the same results as LoRA.
• Authors note the model handles multi-turn QA despite training only on single-turn data, without quantitative evaluation.

Limitations

• Localization is tested on 2-lead recordings and single-lead zero-shot performance drops sharply when the lead carrying the abnormality is masked.
• Multi-ECG and multi-turn evaluations rely on an LLM judge (QwQ-32B) and a small human check of 120 samples; multi-turn ability is not quantified.
• Report generation is compared against zero-shot baselines on label-level AUC, and anyECG-chat does not beat discriminative zero-shot MERL.

Experimental setup

Datasets: MIMIC-ECG PTB-XL ECG-QA European ST-T Database MIT-BIH ST Change Database MIT-BIH Arrhythmia Database

Benchmarks: PTB-XL CPSC CSN MIT-BIH ST Change European ST-T MIT-BIH Arrhythmia MIMIC Multi-ECG QA ECG-QA

Metrics: macro-AUC IoU BLEU ROUGE exact match accuracy LLM score 0-5

Baselines: LLaVa-Med MEIT PULSE ASTCL ST-MEM MERL Unet M3AE

Builds on: LLaVA-1.5 MEIT PULSE MERL

Closest papers by meaning

• From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.89)
• GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.88)
• ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation (NeurIPS 2026 · similarity 0.88)
• Extending Pretrained 10-Second ECG Foundation Models to Longer Horizons (NeurIPS 2026 · similarity 0.87)
• ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation (ICML 2026 · similarity 0.86)
• Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model (ICLR 2025 · similarity 0.86)