Research map / Learning theory and algorithms / Sample complexity
Reliably detecting model failures in deployment without labels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D3M, a label-free, training-data-free deployment monitor that flags deteriorating distribution shifts by comparing the maximum disagreement rate of posterior samples from a variational Bayesian last-layer model against an in-distribution calibration reference.
Problem
Detecting post-deployment model deterioration under distribution shift without labels, without access to training data at deployment, and without flagging non-deteriorating shifts.
Approach
Calibrate a reference distribution of maximum disagreement rates on in-distribution bootstrapped samples by sampling from a variational Bayesian last-layer posterior predictive; at deployment, compute the same statistic on unlabeled deployment batches and flag if it exceeds the (1-alpha) quantile of the reference, with theoretical FPR/TPR analysis.
Key results
- On UCI Heart Disease, CIFAR-10.1 and Camelyon17 at query size 10, D3M reaches TPR .38, .40 and .89 respectively, comparable to the best baselines (Detectron .24, .37, .97).
- At Camelyon17 query size 50, D3M reaches TPR .99 versus .96 for Detectron.
- On UCI Heart Disease at query size 20 D3M's TPR (.25) is lower than Detectron (.57) and CTST (.51), with high variance across methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use D3M instead of Detectron | deployment deterioration monitoring · UCI Heart Disease (query size 10) | TPR: .24$\pm$.04 -> .38$\pm$.19 | Helped |
| Use D3M instead of Detectron | deployment deterioration monitoring · UCI Heart Disease (query size 20) | TPR: .57$\pm$.05 -> .25$\pm$.28 | Hurt |
| Use D3M instead of CTST | deployment deterioration monitoring · UCI Heart Disease (query size 50) | TPR: .98$\pm$.01 -> .69$\pm$.33 | Hurt |
| Use D3M instead of Detectron | deployment deterioration monitoring · CIFAR-10.1 (query size 10) | TPR: .37$\pm$.05 -> .40$\pm$.10 | Helped |
| Use D3M instead of Detectron | deployment deterioration monitoring · CIFAR-10.1 (query size 20) | TPR: .54$\pm$.05 -> .45$\pm$.10 | Hurt |
| Use D3M instead of Detectron | deployment deterioration monitoring · CIFAR-10.1 (query size 50) | TPR: .83$\pm$.04 -> .74$\pm$.12 | Hurt |
| Use D3M instead of Detectron | deployment deterioration monitoring · Camelyon17 (query size 10) | TPR: .97$\pm$.02 -> .89$\pm$.20 | Hurt |
| Use D3M instead of Detectron | deployment deterioration monitoring · Camelyon17 (query size 20) | TPR: 1.0$\pm$.00 -> .93$\pm$.05 | Hurt |
| Use D3M instead of Detectron | deployment deterioration monitoring · Camelyon17 (query size 50) | TPR: .96$\pm$.02 -> .99$\pm$.02 | Helped |
Surprising findings
- D3M's TPR on UCI at query size 20 (.25 ± .28) is below several baselines, and the authors attribute the high variance to sampling noise from posterior sampling.
- The authors report empirically that D3M oversamples hypotheses outside the theoretical class H_p, deviating from the idealized guarantees.
Limitations
- High variance of TPR at small query sizes due to randomness in posterior sampling, especially on UCI.
- Theory provides guarantees only under ground-truth labeling functions with no label noise and only for binary classification.
- Relies on the feature extractor quality; performance on noisy representations is an open question.
Experimental setup
Datasets: UCI Heart Disease CIFAR-10 CIFAR-10.1 Camelyon17 (WILDS) GEMINI
Benchmarks: UCI Heart Disease CIFAR-10.1 Camelyon17 GEMINI
Metrics: true positive rate false positive rate
Baselines: BBSD Relative Mahalanobis distance Deep Ensemble CTST MMD-D H-divergence Detectron Domain Classifier
Builds on: Variational Bayesian Last Layers Detectron Harrison et al. 2024 VBLL
Closest papers by meaning
- Statistically Valid Post-Deployment Monitoring Should Be Standard for AI-Based Digital Health (NeurIPS 2025 · similarity 0.86)
- Training ML Models with Predictable Failures (NeurIPS 2026 · similarity 0.83)
- PICID: A Modular Evaluation Infrastructure for Reproducible PHM Across Tasks and Domains (NeurIPS 2026 · similarity 0.82)
- Sequential Harmful Shift Detection Without Labels (NeurIPS 2024 · similarity 0.82)
- Monitoring Risks in Test-Time Adaptation (NeurIPS 2025 · similarity 0.82)
- Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.