Research map / Vision-language and multimodal / Human and embodied modeling
DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DAMPER, a dual-stage radiology report generator that aligns CXR features with MeSH via GAN (coarse stage) and matches image patches to report phrases via hypergraph matching (fine stage), reporting gains on IU-Xray and MIMIC-CXR.
Problem
Automatic medical report generation from chest X-rays suffers from skipping the physician's clinical workflow (overall impression then detailed examination) and from misaligned token-level image-word matching that fragments semantics across patches and words.
Approach
Mimic radiologist workflow with two stages: a GAN-based MeSH-CXR alignment that generates coarse MeSH representations, followed by hypergraph construction over image patches and report phrases with node and hypergraph-level matching losses to capture intra-modal high-order relations before report decoding.
Key results
- On IU-Xray, DAMPER reaches BLEU-4 0.225 and METEOR 0.284, versus 0.212 and 0.242 for the best prior baselines shown in the table (MedM2G BLEU-4, KiUT METEOR).
- On MIMIC-CXR, DAMPER achieves BLEU-4 0.193 and METEOR 0.289 versus 0.142 (MedM2G) and 0.161 (MMTN) for the strongest baselines in the table.
- On MIMIC-CXR CE metrics, DAMPER gets F1 0.507 versus 0.476 for PromptMRG; its recall (0.473) is below PromptMRG's 0.509.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MCG and HFG dual-stage alignment | DAMPER · IU-Xray and MIMIC-CXR test sets | BLEU-4: IU-Xray: 0.212 -> 0.225; MIMIC-CXR: 0.142 -> 0.193 | Helped |
| Add CMG (ablation) | DAMPER · IU-Xray test | METEOR: 0.193 -> 0.284 | Helped |
| Add MCA (ablation) | DAMPER · IU-Xray test | METEOR: 0.186 -> 0.284 | Helped |
| Add Intra-RCA (ablation) | DAMPER · IU-Xray test | BLEU-1: 0.507 -> 0.520 | Helped |
| Add Inter-RCA (ablation) | DAMPER · IU-Xray test | BLEU-1: 0.511 -> 0.520 | Helped |
| Use ground-truth MeSH instead of CheXpert disease labels (ablation) | DAMPER · IU-Xray test | METEOR: 0.187 -> 0.284 | Helped |
| Use hypergraphs instead of graphs (ablation) | DAMPER · IU-Xray test | METEOR: 0.204 -> 0.284 | Helped |
| Add the complete MCG stage rather than omit it (ablation) | DAMPER · IU-Xray test | BLEU-1: 0.471 -> 0.520 | Helped |
Surprising findings
- Removing the Intra-RCA module hurts more than removing Inter-RCA, and on IU-Xray the w/o CMG variant scores lower than w/o MCG on several metrics.
- On the zero-shot MIMIC-ABN test, DAMPER is below ICON on BLEU-1 and ROUGE-L (0.316 vs 0.337 and 0.209 vs 0.236).
Limitations
- Multi-view visual information is needed for accurate alignment; some views give incomplete pathology-region correspondence.
- MIMIC-CXR MeSH is generated by the CheXpert labeler rather than given, and ground-truth MeSH is needed on IU-Xray; reliance on MeSH quality.
- Recall on MIMIC-CXR CE metrics is lower than PromptMRG, and several metrics trail the best baselines.
Experimental setup
Datasets: IU-Xray MIMIC-CXR MIMIC-ABN
Benchmarks: IU-Xray MIMIC-CXR MIMIC-ABN
Metrics: BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L CE precision CE recall CE F1
Baselines: AlignTransformer R2Gen R2GenCMN PPKED KiUT MMTN COMG MedM2G PromptMRG M2KT CvT2DistilGPT2 RECAP
Builds on: R2Gen RGM Clinical-BERT CheXpert labeler
Closest papers by meaning
- DART: Disease-aware Image-Text Alignment and Self-correcting Re-alignment for Trustworthy Radiology Report Generation (CVPR 2025 · similarity 0.88)
- Automatic Radiology Reports Generation via Memory Alignment Network (AAAI 2024 · similarity 0.88)
- S2D-Align: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation (AAAI 2026 · similarity 0.87)
- Image-aware Evaluation of Generated Medical Reports (NeurIPS 2024 · similarity 0.87)
- Bootstrapping Large Language Models for Radiology Report Generation (AAAI 2024 · similarity 0.86)
- PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.