Research map / Label-efficient and robust learning / OOD and anomaly detection
Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
Chunjong Park, Anas Awadalla, Tadayoshi Kohno, Shwetak Patel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Mahalanobis or Gram-matrix OOD detection instead of unfiltered model predictions | Pretrained health classifiers · Open-set datasets: Face, CIFAR10, AudioSet, MotionSense, mHealth | TNR @ TPR95: Mahalanobis: 97.96–100.00; Gram: 95.01–100.00 | Improved |
| Use Mahalanobis or Gram-matrix detection on near-distribution inputs | Pretrained health classifiers · ISIC 2017, Stethoscope, Kaggle Parkinson's | TNR @ TPR95: 10.13; 25.90; 45.60; 41.77; 98.00; 98.96 | Mixed |
| Use energy scores without fine-tuning for OOD detection | Pretrained health classifiers · Health OOD datasets | TNR @ TPR95: 0.00; 5.06; 8.12; 7.29; 70.00 | Hurt |
| Expose a human-readable confidence score derived from OOD scores | Health diagnostic results shown to study participants · User study, all conditions | User-perceived trustworthiness effect size: 0.393 | Improved |
| Expose a human-readable confidence score | Health diagnostic results shown to study participants · User study, all conditions | Effect size for impact on medical decisions: 0.178 | Improved |
| Show confidence scores alongside image, audio, or motion diagnostic outputs | Health diagnostic results shown to study participants · User study, image/audio/motion | Effect size for user-perceived trustworthiness: Image 0.436; Audio 0.384; Motion 0.361 | Mixed |
| Show confidence scores alongside image, audio, or motion diagnostic outputs | Health diagnostic results shown to study participants · User study, image/audio/motion | Effect size for medical decisions: Image 0.225; Audio 0.173; Motion 0.133 | Mixed |
| Use a confidence score to help participants distinguish inputs the model can interpret | Health application users · User study, image/audio/motion | Jaccard index for selecting high-confidence inputs: 0.75, 0.66, and 0.64 | Improved |
| Evaluate existing health classifiers on shifted rather than in-distribution data | Skin lesion, lung sound, and Parkinson's classifiers · Near-distribution OOD datasets | classification accuracy: 18% to 76% | Hurt |
Closest papers by meaning
- Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (NeurIPS 2019 · similarity 0.85)
- DIsoN: Decentralized Isolation Networks for Out-of-Distribution Detection in Medical Imaging (NeurIPS 2025 · similarity 0.83)
- OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detection (CVPR 2025 · similarity 0.83)
- A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis (NeurIPS 2024 · similarity 0.82)
- Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness (ICLR 2024 · similarity 0.82)
- ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms (ICLR 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
