Research map / Architectures and efficiency / RNNs and normalization
CheXwhatsApp: A Dataset for Exploring Challenges in the Diagnosis of Chest X-rays through Mobile Devices
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CheXwhatsApp, a dataset of 141,804 paired original and WhatsApp-compressed chest X-rays built from NIH and RSNA, and benchmarks prediction instability, out-of-lung saliency, and localization instability for CXR classifiers and detectors.
Problem
AI-based chest X-ray diagnosis on images shared via WhatsApp suffers from compression artifacts that cause prediction changes, saliency outside the lungs, and localization changes, and no paired dataset exists to study these challenges.
Approach
Create paired original and WhatsApp-compressed CXR images by automatically sending NIH and RSNA images through WhatsApp, define metrics (PI, OLS, LI scores) for prediction, saliency and localization instability, and show that training on paired data reduces these instabilities.
Key results
- Models trained on NIH-WA have up to 80% lower PI Score than those trained on NIH (e.g., DenseNet 15.74 to 3.31).
- Training on RSNA-WA reduces localization instability by up to 81% (YOLOv8s LI Score 4.02 to 0.78) compared with RSNA.
- OLS scores drop substantially for models trained on NIH-WA, e.g., DenseNet GradCAM original OLS from 18.79 to 12.53.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on CheXwhatsApp (NIH-WA) paired original+compressed data instead of NIH only | DenseNet, InceptionNet, ResNet-50, ResNeXt-50 classifiers · NIH test, original and WhatsApp images | PI Score (lower is better): 77-79% lower PI Score | Helped |
| train on CheXwhatsApp (RSNA-WA) paired data instead of RSNA only | YOLOv5s, YOLOv8s, Faster-RCNN · RSNA test, original and WhatsApp images, t=0.7 | LI Score (lower is better): YOLOv5s -72%, YOLOv8s -81%, Faster-RCNN -13% | Mixed |
| train on NIH-WA paired data instead of NIH only | DenseNet, Inception, ResNet, ResNeXt attribution maps · NIH test, OLS Score with GradCAM, GradCAM++, EigenCAM | OLS Score (lower is better): reductions of roughly 1-82% depending on model and CAM method; EigenCAM gains are small (1-3%) for Inception, ResNet, Re | Helped |
| train on JPEG-compressed CXR images instead of original images (ablation) | classification and detection models · JPEG-compressed test sets (quality 20-90) | PI, OLS, LI scores: PIP, OLS and LIP further increased | Hurt |
| train on CheXwhatsApp instead of original images (ablation) | classifiers evaluated on JPEG-compressed test images · JPEG quality factors 20, 40, 60, 80, 90 | PI Score and LI Score: average 26% lower prediction instability and 10% lower localization instability | Helped |
| train on NIH-WA paired data for Faster-RCNN localization | Faster-RCNN on RSNA · RSNA test, LI Score t=0.7 | LI Score (lower is better): only 13% reduction (12.43 to 10.86), far smaller than YOLO gains | Mixed |
Surprising findings
- Models trained on JPEG-compressed CXR images showed even higher PIP, OLS and LIP than the original-trained models.
- Out-of-lung saliency is worse in WhatsApp-compressed images, and for EigenCAM the OLS scores remain above 94% for all models even after training on NIH-WA.
Limitations
- Dataset is derived from existing NIH and RSNA datasets rather than newly collected clinical data.
- The new dataset was not used to create or improve an actual mHealth application.
- WhatsApp Business API incurred per-compression charges and the pipeline required manual device coordination.
Experimental setup
Datasets: NIH ChestX-ray14 RSNA pneumonia detection
Benchmarks: NIH NIH-WA RSNA RSNA-WA
Metrics: PI Score OLS Score LI Score IOU-based localization agreement
Baselines: DenseNet ResNet ResNeXt Inception Faster R-CNN YOLOv5s YOLOv8s
Builds on: GradCAM Faster R-CNN YOLOv5 YOLOv8
Closest papers by meaning
- MedRAX: Medical Reasoning Agent for Chest X-ray (ICML 2025 · similarity 0.84)
- CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning (CVPR 2025 · similarity 0.83)
- CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays (NeurIPS 2025 · similarity 0.83)
- From Complexity to Clarity: Transforming Chest X-ray Reports with Chained Prompting (Student Abstract) (AAAI 2025 · similarity 0.82)
- CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset (CVPR 2025 · similarity 0.82)
- Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.