Research map / Vision-language and multimodal / Multimodal LLMs

ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ECG-R1, a reasoning ECG multimodal LLM trained with protocol-guided synthetic CoT data, decoupled signal/image encoders with interleaved modality dropout, and RL with evidence rewards (EDER) to improve reliable ECG interpretation under modality missing.

Problem

Existing multimodal LLMs produce plausible but clinically incorrect ECG interpretations (hallucinations), and ECG omni-perception models degrade and become cross-modally inconsistent when either the ECG signal or image is missing.

Approach

Generate ECG interpretation supervision by feeding deterministic signal-derived features plus a monograph-derived five-phase protocol to an LLM; use decoupled encoders for signal and image with interleaved modality dropout and token-block swapping during training; and add a stepwise evidence-coverage reward alongside accuracy and format rewards in DAPO-based RL.

Key results

• ECG-R1 (RL) reaches 80.29 diagnosis accuracy on the ECG-Grounding test set versus 74.70 for GEM and 31.48 for GPT-5.1-Instant (DeepSeek judge).
• Under image-missing (time-series only) evaluation, IMD raises diagnosis accuracy from 36.77 to 77.91.
• Cardiologist ratings (n=100 cases, 4 cardiologists) give ECG-R1 4.34 analytical accuracy vs 3.89 for GEM.

Surprising findings

• Proprietary MLLMs such as GPT-5.1 and Gemini-3-Pro achieve diagnosis accuracy below 35% on the ECG-Grounding test set, with severe hallucinations widespread.
• On ECG-Bench abnormality detection, ECG-R1 is not explicitly optimized for it yet is competitive, reaching state of the art on CSN and G12EC but trailing GEM on CPSC 2018 F1 (51.2 vs 61.1).

Limitations

• Reward and evaluation rely on LLM judges (DeepSeek-V3.1-Terminus, GLM-5) and a 100-case cardiologist subset.
• Abnormality detection is not the model's primary target and shows gaps on some datasets such as CPSC 2018.
• Model outputs may contain errors; authors state research-only use requiring clinical oversight.

Experimental setup

Datasets: MIMIC-IV-ECG ECGInstruct ECG-Grounding test set

Benchmarks: ECG-Grounding test set ECG-Bench

Metrics: diagnosis accuracy analysis completeness analysis relevance lead evidence validity ECG feature grounding evidence-based reasoning clinical diagnostic fidelity BLEU-4 ROUGE-L SBERT-Score

Baselines: GEM PULSE GPT-5.1-Instant Gemini-3-Pro MedGemma-27B QoQ-Med-VL-7B Qwen3-VL-8B-Instruct

Builds on: DAPO DeepSeek-R1 R1-VL GEM ECG-Grounding

Closest papers by meaning

• ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation (NeurIPS 2026 · similarity 0.92)
• GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.90)
• anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task Understanding (AAAI 2026 · similarity 0.86)
• CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations (NeurIPS 2026 · similarity 0.86)
• Robustness and Visual Explanation for Black Box Image, Video, and ECG Signal Classification with Reinforcement Learning (AAAI 2024 · similarity 0.86)
• CMR-RD: Long-Tailed Adaptive VLM for Explainable CMR Diagnosis (CVPR 2026 · similarity 0.86)