Research map / Vision-language and multimodal / CLIP and zero-shot
PairAug: What Can Augmented Image-Text Pairs Do for Radiology?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PairAug, which jointly augments radiology image-report pairs via LLM-generated reports with text-to-image synthesis (InterAug) and cross-attention-swapped intra-patient image editing (IntraAug), plus CLIP-based pruning, to improve medical VLP on CheXpert, PadChest and RSNA.
Problem
Medical vision-language pre-training needs large paired image-report data, which is scarce in radiology; existing augmentation methods expand only images or only text, breaking image-text semantic correspondence.
Approach
Augment image and text together: generate new patient cases from LLM-rewritten reports (InterAug), and generate varied conditions for the same patient by swapping diffusion cross-attention maps between original and edited reports (IntraAug), then prune with MedCLIP-based consistency scores.
Key results
- PairAug reaches 88.34% mean AUC on CheXpert zero-shot and 70.79% on PadChest, about 2.10 and 4.50 points above CheXzero-S without ensemble.
- On RSNA with 1% labels, PairAug achieves 85.89% AUC versus 83.45% for CheXzero-S base.
- Two radiology residents identified synthetic images at 61% and synthetic reports at 50% accuracy, near chance, in a visual Turing test.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add paired image-text augmentation | CheXzero-S · ChestXpert zero-shot | AUC: 86.24 -> 88.34 | Helped |
| Add paired image-text augmentation | CheXzero-S · PadChest zero-shot | AUC: 66.29 -> 70.79 | Helped |
| Add paired image-text augmentation | CheXzero-S · RSNA 1% labeled data | AUC: 83.45 -> 85.89 | Helped |
| Add paired image-text augmentation | CheXzero-S · RSNA 100% labeled data | AUC: 86.99 -> 88.63 | Helped |
| Add image-only augmentation | CheXzero-S · ChestXpert zero-shot | AUC: 86.24 -> 87.43 | Helped |
| Add text-only augmentation | CheXzero-S · ChestXpert zero-shot | AUC: 86.24 -> 87.03 | Helped |
| Add InterAug without data pruning (ablation) | CheXzero-S · ChestXpert zero-shot | AUC: 86.24 -> 84.91 | Hurt |
| Add Pra pruning to InterAug (ablation) | CheXzero-S with InterAug · ChestXpert zero-shot | AUC: 84.91 -> 87.36 | Helped |
| Add IntraAug without data pruning (ablation) | CheXzero-S · ChestXpert zero-shot | AUC: 86.24 -> 85.79 | Hurt |
| Add Pre pruning to IntraAug (ablation) | CheXzero-S with IntraAug · ChestXpert zero-shot | AUC: 85.79 -> 87.30 | Helped |
| Add both augmentation branches and both pruning methods (ablation) | CheXzero-S · ChestXpert zero-shot | AUC: 86.24 -> 88.34 | Helped |
Surprising findings
- Adding InterAug or IntraAug alone without pruning gave limited or even decreased performance (InterAug alone -1.59 average change), with gains appearing mainly after pruning.
Limitations
- Performance depends heavily on the quality of the RoentGen generator, which is trained only on chest X-rays and is biased and lacks diversity.
- ChatGPT sometimes produces informal terms in generated reports.
- Synthetic report-image alignment judged 'good' only 61% of the time by radiologists.
Experimental setup
Datasets: MIMIC-CXR CheXpert PadChest RSNA Pneumonia
Benchmarks: CheXpert PadChest RSNA Pneumonia
Metrics: AUC accuracy F1
Baselines: ImageNet pre-training MGCA CXR-CLIP CheXzero AugImg AugText AugText+AugImg
Builds on: CheXzero CLIP RoentGen Prompt-to-Prompt attention control MedCLIP
Closest papers by meaning
- Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.86)
- RadZero: Similarity-Based Cross-Attention for Explainable Vision-Language Alignment in Chest X-ray with Zero-Shot Multi-Task Capability (NeurIPS 2025 · similarity 0.85)
- CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification (CVPR 2024 · similarity 0.85)
- S2D-Align: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation (AAAI 2026 · similarity 0.84)
- G2D: From Global to Dense Radiography Representation Learning via Vision-Language Pre-training (NeurIPS 2024 · similarity 0.84)
- BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.