Research map / Label-efficient and robust learning / Self-supervised and contrastive
CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CheXWorld, a JEPA-style self-supervised world model for chest radiographs that jointly learns local anatomy, global layout, and domain-variation transitions, pre-trained on about 0.5M chest X-rays.
Problem
Self-supervised representation learning for radiographs that captures medical knowledge such as anatomy and appearance-domain variation, which existing SSL methods and medical foundation models handle poorly.
Approach
Frames radiograph SSL as world modeling in latent space: a predictor forecasts target patch embeddings from a context conditioned on relative spatial displacement and augmentation parameters, unifying local masked reconstruction, global relative-position prediction, and domain-transition prediction in one forward pass.
Key results
- CheXWorld reaches 95.24 AUROC on VinDr-CXR classification, exceeding the best listed alternative (SimMIM 92.81 / MAE 92.76), and 84.58 dice on SIIM-ACR segmentation vs. 83.60 for Adam-v2.
- With 10% of VinDr-CXR training data, CheXWorld outperforms all baselines trained on the full dataset, per the paper's sample-efficiency claim.
- Domain sensitivity test gives top-5 recall of 77.67 versus 7.81 for random choice; removing the domain condition drops it to 44.37.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pre-train with CheXWorld rather than the strongest listed baseline | ViT-B radiograph representation learning · ShenZhen | AUROC: 98.09±0.13 -> 98.88±0.06 | Helped |
| Pre-train with CheXWorld rather than the strongest listed baseline | ViT-B radiograph representation learning · RSNA (CLS) | accuracy: 74.09±0.39 -> 75.03±0.39 | Helped |
| Pre-train with CheXWorld rather than the strongest listed baseline | ViT-B radiograph representation learning · SIIM-ACR | Dice score: 83.60±0.33 -> 84.58±0.34 | Helped |
| Pre-train with CheXWorld rather than the strongest listed baseline | ViT-B radiograph representation learning · MedFMC-ChestDR 1-shot | AUROC: 61.31±0.27 -> 64.60±1.00 | Helped |
| Add the domain condition (ablation) | CheXWorld · Domain variation sensitivity test | Recall@5: 44.37 -> 77.67 | Helped |
| Add the relative-position latent variable (ablation) | CheXWorld global anatomical layout modeling · VinDr-CXR 1% training data | AUROC: 89.12±0.76 -> 90.53±1.01 | Helped |
| Add the relative-position latent variable (ablation) | CheXWorld global anatomical layout modeling · RSNA 1% training data | accuracy: 68.37±0.50 -> 70.01±0.33 | Helped |
| Add the domain-transition condition (ablation) | CheXWorld domain variation modeling · VinDr-CXR 1% training data | AUROC: 89.91±0.60 -> 90.53±1.01 | Helped |
| Add the domain-transition condition (ablation) | CheXWorld domain variation modeling · RSNA 1% training data | accuracy: 68.98±0.60 -> 70.01±0.33 | Helped |
| Use the unified formulation rather than a naive combination of losses (ablation) | CheXWorld · VinDr-CXR 1% training data | AUROC: 87.60±0.72 -> 90.53±1.01 | Helped |
| Use the unified formulation rather than a naive combination of losses (ablation) | CheXWorld · RSNA 1% training data | accuracy: 66.56±0.39 -> 70.01±0.33 | Helped |
Surprising findings
- Rad-DINO, trained on 142M LVD images plus private X-rays with about 20x more compute, performs comparably to CheXWorld rather than clearly above it.
Limitations
- Pre-training uses only about 0.5M frontal chest X-rays, and evaluation is limited to chest radiograph benchmarks.
- The three modeled aspects are acknowledged as not covering every dimension of medical understanding.
- Rad-DINO is included only for reference because it is not directly comparable in data and compute.
Experimental setup
Datasets: chest X-ray pre-training corpus (~0.5M frontal radiographs from public datasets) VinDr-CXR ShenZhen-CXR NIH ChestX-ray14 CheXpert RSNA Pneumonia MedFMC-ChestDR SIIM-ACR Pneumothorax
Benchmarks: VinDr-CXR ShenZhen-CXR ChestX-ray14 RSNA pneumonia classification CheXpert SIIM-ACR Pneumothorax RSNA pneumonia segmentation MedFMC-ChestDR few-shot
Metrics: AUROC accuracy dice score recall@k
Baselines: MoCo-v3 DINO BEiT LVM-Med MAE SimMIM Adam-v2 Rad-DINO
Builds on: I-JEPA MAE RCDM
Closest papers by meaning
- A World Model of Radiologist Reading for Medical Image Representation Learning (NeurIPS 2026 · similarity 0.87)
- CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays (NeurIPS 2025 · similarity 0.86)
- Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models (NeurIPS 2026 · similarity 0.85)
- G2D: From Global to Dense Radiography Representation Learning via Vision-Language Pre-training (NeurIPS 2024 · similarity 0.85)
- BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays (NeurIPS 2024 · similarity 0.85)
- AOR: Anatomical Ontology-Guided Reasoning for Medical Large Multimodal Model in Chest X-Ray Interpretation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.