AI Paper Atlas

Research map / Label-efficient and robust learning / OOD and anomaly detection

Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection

Chunjong Park, Anas Awadalla, Tadayoshi Kohno, Shwetak Patel

NeurIPS 2021 · Label-efficient and robust learning › OOD and anomaly detection · approach: Uncertainty and misclassification detection · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Apply Mahalanobis or Gram-matrix OOD detection instead of unfiltered model predictionsPretrained health classifiers · Open-set datasets: Face, CIFAR10, AudioSet, MotionSense, mHealthTNR @ TPR95: Mahalanobis: 97.96–100.00; Gram: 95.01–100.00Improved
Use Mahalanobis or Gram-matrix detection on near-distribution inputsPretrained health classifiers · ISIC 2017, Stethoscope, Kaggle Parkinson'sTNR @ TPR95: 10.13; 25.90; 45.60; 41.77; 98.00; 98.96Mixed
Use energy scores without fine-tuning for OOD detectionPretrained health classifiers · Health OOD datasetsTNR @ TPR95: 0.00; 5.06; 8.12; 7.29; 70.00Hurt
Expose a human-readable confidence score derived from OOD scoresHealth diagnostic results shown to study participants · User study, all conditionsUser-perceived trustworthiness effect size: 0.393Improved
Expose a human-readable confidence scoreHealth diagnostic results shown to study participants · User study, all conditionsEffect size for impact on medical decisions: 0.178Improved
Show confidence scores alongside image, audio, or motion diagnostic outputsHealth diagnostic results shown to study participants · User study, image/audio/motionEffect size for user-perceived trustworthiness: Image 0.436; Audio 0.384; Motion 0.361Mixed
Show confidence scores alongside image, audio, or motion diagnostic outputsHealth diagnostic results shown to study participants · User study, image/audio/motionEffect size for medical decisions: Image 0.225; Audio 0.173; Motion 0.133Mixed
Use a confidence score to help participants distinguish inputs the model can interpretHealth application users · User study, image/audio/motionJaccard index for selecting high-confidence inputs: 0.75, 0.66, and 0.64Improved
Evaluate existing health classifiers on shifted rather than in-distribution dataSkin lesion, lung sound, and Parkinson's classifiers · Near-distribution OOD datasetsclassification accuracy: 18% to 76%Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy