Research map / Architectures and efficiency / Pruning and efficient tuning
H-Tuning: Toward Low-Cost and Efficient ECG-based Cardiovascular Disease Detection with Pre-Trained Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H-Tuning, a joint workflow combining mix-order (SPSA plus first-order gradient refinement) low-rank fine-tuning of a pre-trained ECG model with knowledge distillation into tiny student models for low-cost CVD detection.
Problem
Fine-tuning and inference of large pre-trained ECG models are too costly in GPU memory and latency for low-level clinical and wearable devices, and no joint framework addresses both while preserving CVD detection performance under low budgets.
Approach
Refines SPSA zeroth-order gradient estimates with first-order gradients computed on a tiny batch (with norm normalization), combines this with LoRA, updates deep layers with backpropagation and shallow layers with the mix-order method, then distills the fine-tuned model into tiny multi-lead student networks.
Key results
- H-Tuning reduces GPU memory during fine-tuning by 6.02 to 6.34 times versus LoRA and Full FT while achieving macro F2 within 0.5% (Full FT) and 0.7% (LoRA) on average (0.600 average).
- Distilled 12-lead student model cuts inference time by 4.52x, GPU memory by 19.83x, and parameters by 194.23x relative to the 50.5M-parameter teacher (0.26M parameters, 1.28 ms, 18.05 MB).
- Student models with 12-lead ECG reach similar or better macro F2 than the H-Tuning teacher (e.g., 0.628 vs 0.628 on PTB-XL style comparisons; teacher average 0.586/0.628/0.550/0.634 across datasets).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use H-Tuning instead of Full FT for fine-tuning | Zhou et al. (2024) pre-trained model · G12EC, PTB-XL, Ningbo, and Chapman | GPU memory consumption: 6.34 times | Helped |
| Use H-Tuning instead of Full FT | Zhou et al. (2024) pre-trained model · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2: 0.5% | No clear effect |
| Use H-Tuning instead of LoRA | Zhou et al. (2024) pre-trained model · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2: 0.7% | No clear effect |
| Distill the H-Tuning teacher into a student model | H-Tuning teacher · ECG inference | Inference latency: 4.52 times | Helped |
| Distill the H-Tuning teacher into a student model | H-Tuning teacher · ECG inference | GPU memory consumption: 19.83 times | Helped |
| Add SPSA gradient estimation to H-Tuning (ablation) | H-Tuning · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2: 0.575 -> 0.586; 0.616 -> 0.628; 0.525 -> 0.550; 0.612 -> 0.634 | Helped |
| Add gradient refinement to H-Tuning (ablation) | H-Tuning · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2: 0.557 -> 0.586; 0.581 -> 0.628; 0.535 -> 0.550; 0.580 -> 0.634 | Helped |
| Add gradient normalization to H-Tuning (ablation) | H-Tuning · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2: 0.588 -> 0.586; 0.609 -> 0.628; 0.536 -> 0.550; 0.596 -> 0.634 | Mixed |
| Add low-rank adaptation to H-Tuning (ablation) | H-Tuning · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2: 0.564 -> 0.586; 0.588 -> 0.628; 0.527 -> 0.550; 0.603 -> 0.634 | Helped |
| Increase rank r from 4 to 16 (ablation) | H-Tuning · G12EC, PTB-XL, Ningbo, and Chapman | Macro Fβ=2 | Helped |
| Increase gradient-refinement batch size N1 (ablation) | H-Tuning · Ningbo and G12EC | Macro Fβ=2 | Helped |
| Increase gradient-refinement batch size N1 (ablation) | H-Tuning · PTB-XL and Chapman | Macro Fβ=2 | No clear effect |
Surprising findings
- Student models sometimes outperform their teacher models, which the authors attribute to strong teachers not always being better teachers.
- MeZO and MeZO+LoRA collapse to near-chance-level performance (macro AUC around 0.5) without prompt engineering on ECG data.
Limitations
- Evaluated only on ECG-based CVD detection; generalization to other physiological signals is left for future work.
- Zeroth-order gradient estimate is biased with n=1 and small perturbation scale; theoretical convergence not established in the main text.
- Hardware used for experiments is not specified.
Experimental setup
Datasets: Chapman-Shaoxing G12EC PTB-XL Ningbo CODE wearable 12-lead ECG external dataset (Lai et al., 2023)
Benchmarks: Chapman-Shaoxing G12EC PTB-XL Ningbo wearable 12-lead ECG external validation
Metrics: macro AUC macro F-beta (beta=2) peak GPU memory inference time number of parameters ranking loss coverage MAP macro G-beta
Baselines: Full FT LoRA Linear Probing MeZO MeZO + LoRA Addax Addax + LoRA LoHO LoHO + LoRA
Builds on: LoRA MeZO Addax LoHO knowledge distillation
Closest papers by meaning
- Extending Pretrained 10-Second ECG Foundation Models to Longer Horizons (NeurIPS 2026 · similarity 0.85)
- From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining (ICML 2025 · similarity 0.84)
- Region-Disentangled Diffusion Model for High-Fidelity PPG-to-ECG Translation (AAAI 2024 · similarity 0.84)
- ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation (NeurIPS 2026 · similarity 0.84)
- GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images (NeurIPS 2025 · similarity 0.83)
- Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.