Research map / Large language models / Language, safety and interpretability
LLM-RG4: Flexible and Factual Radiology Report Generation Across Diverse Input Contexts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MIMIC-RG4, a dataset of radiology reports reconstructed to match four input scenarios, and LLM-RG4, a Vicuna-7B-based report generator with adaptive token fusion and token-level loss weighting.
Problem
Radiology report generation models assume a fixed single-image input, causing a mismatch with reports that reference comparisons, procedures, or views, and leading to input-agnostic hallucinations when inputs vary.
Approach
Reconstruct MIMIC-CXR reports so each matches its input scenario (single/multi-view, with/without prior report), and use an adaptive token fusion module that concatenates modality features along the feature dimension to keep a fixed token count, plus token-level loss weights from CheXbert labels and Integrated Gradients to emphasize positive and uncertain diagnoses.
Key results
- On MIMIC-RG4 single-view no-longitudinal setting, LLM-RG4 reaches F1 0.609 and BLEU-4 0.196 versus cxrmate's 0.566 and 0.122.
- On MIMIC-CXR, LLM-RG4 achieves F1 0.588 versus 0.548 for Promptmrg and 0.553 for MAIRA-1 (7B).
- LLM-RG4 has an input-agnostic hallucination rate of 1.5% on MIMIC-CXR versus 54.9% for CheXagent, and 0.2% to 1.4% across MIMIC-RG4 settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM-RG4 instead of Promptmrg | MIMIC-CXR sn report generation · MIMIC-CXR sn | F1: 0.548 -> 0.588 | Helped |
| Use LLM-RG4 instead of cxrmate | MIMIC-RG4 report generation · MIMIC-RG4 sn | F1: 0.566 -> 0.609 | Helped |
| Use LLM-RG4 instead of cxrmate | MIMIC-RG4 report generation · MIMIC-RG4 sw | F1: 0.561 -> 0.610 | Helped |
| Use LLM-RG4 instead of cxrmate | MIMIC-RG4 report generation · MIMIC-RG4 mn | F1: 0.533 -> 0.559 | Helped |
| Use LLM-RG4 instead of cxrmate | MIMIC-RG4 report generation · MIMIC-RG4 mw | F1: 0.523 -> 0.563 | Helped |
| Add TLW (ablation) | LLM-RG4 with interleaved inputs on MIMIC-RG4 · MIMIC-RG4, stage 2 average | F1-14: 0.551 -> 0.577 | Helped |
| Add TLW (ablation) | LLM-RG4 with ATF on MIMIC-RG4 · MIMIC-RG4, stage 2 average | F1-14: 0.556 -> 0.585 | Helped |
| Add TLW (ablation) | LLM-RG4 with ATF on MIMIC-RG4 · MIMIC-RG4, stage 2 average | BLEU-4: 0.205 -> 0.199 | Hurt |
| Add ATF to interleaved inputs (ablation) | LLM-RG4 on MIMIC-RG4 · MIMIC-RG4, stage 2 average | BLEU-4: 0.201 -> 0.205 | Helped |
| Train jointly across settings rather than on each setting alone (ablation) | LLM-RG4 · MIMIC-RG4 mn | F1: 0.528 -> 0.559 | Helped |
| Train jointly across settings rather than on each setting alone (ablation) | LLM-RG4 · MIMIC-RG4 mw | F1: 0.531 -> 0.563 | Helped |
| Train jointly across settings rather than on each setting alone (ablation) | LLM-RG4 · MIMIC-RG4 sn | F1: 0.599 -> 0.609 | Helped |
Surprising findings
- Ground-truth MIMIC-CXR reports show an 88.5% input-agnostic hallucination score under the original-NLG setting, yet LLM-RG4 remains competitive on original-report NLG metrics.
Limitations
- Model sometimes omits diagnoses such as pneumonia and suspects pneumothorax in the mn scenario, showing inconsistency across input settings.
- Dataset depends on LLM-generated reconstructions and a BERT discriminator trained on Llama3 pseudo-labels.
- Only MIMIC-CXR is used, a single public dataset.
Experimental setup
Datasets: MIMIC-CXR MIMIC-RG4
Benchmarks: MIMIC-CXR MIMIC-RG4
Metrics: micro precision micro recall F1 (CheXbert) BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L input-agnostic hallucination rate (DiscBERT)
Baselines: R2Gen R2GenCMN CVT2Dis. KiUT RGRG EKAGen Promptmrg R2GenGPT CheXagent MAIRA-1 Med-PaLM R2-LLM
Builds on: LLaVA Perceiver CheXbert Integrated Gradients CXRMate
Closest papers by meaning
- Bootstrapping Large Language Models for Radiology Report Generation (AAAI 2024 · similarity 0.92)
- HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generation (AAAI 2025 · similarity 0.90)
- Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation (NeurIPS 2026 · similarity 0.87)
- S2D-Align: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation (AAAI 2026 · similarity 0.87)
- CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation (CVPR 2026 · similarity 0.87)
- Radiology Report Generation via Multi-objective Preference Optimization (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.