Research map / Label-efficient and robust learning / Test-time adaptation
Tracking the risk of a deployed model and detecting harmful distribution shifts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use variance-adaptive source confidence bounds instead of Hoeffding bounds | Sequential risk monitoring · Simulated label shift | target samples needed to detect harmful risk increase | Helped |
| Use betting-based confidence bounds rather than the other evaluated bounds | Sequential risk monitoring · Simulated label shift | detection power | Helped |
| Use betting-based lower confidence bounds instead of CM-EB bounds | Sequential risk monitoring · Simulated i.i.d. target data | target-risk lower-bound tightness | Mixed |
| Use CM-EB bounds to estimate running risk under distribution drift | Sequential risk monitoring · Simulated drifting target distribution | harmful risk-increase detection | Helped |
| Monitor a prespecified risk increase rather than test for any distribution shift | Deployed model monitoring · MNIST-C | misclassification risk detection | Helped |
| Apply fog corruption to clean CIFAR-10 images | ResNet-32 set-valued predictor · CIFAR-10-C | miscoverage risk | Mixed |
| Increase the prescribed miscoverage level, producing larger prediction sets | CIFAR-10-C set-valued predictor · CIFAR-10-C | coverage degradation under fog | Helped |
| Continuously monitor with a fixed-time CLT lower bound without multiple-testing correction | Sequential risk monitoring · Simulated Bernoulli data | false alarm rate | Hurt |
| Correct fixed-time CLT bounds for repeated testing using Bonferroni allocation | Sequential risk monitoring · Simulated Bernoulli data | detection power | Hurt |
| Use a conformal test martingale to detect a harmful shift from a cold start | Conformal test martingale · Simulated harmful label shift | harmful-shift detection | Hurt |
| Warm-start the conformal test martingale with source-distribution examples | Conformal test martingale · Simulated harmful label shift | harmful-shift detection | Helped |
| Use conformal test martingales to detect slow drift rather than sharp drift | Conformal test martingale · Simulated gradual distribution drift | detection power | Hurt |
Closest papers by meaning
- Sequential Harmful Shift Detection Without Labels (NeurIPS 2024 · similarity 0.92)
- Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias (NeurIPS 2026 · similarity 0.89)
- Monitoring Risks in Test-Time Adaptation (NeurIPS 2025 · similarity 0.87)
- When to retrain a machine learning model (ICML 2025 · similarity 0.86)
- "What is Different Between These Datasets?" A Framework for Explaining Data Distribution Shifts (NeurIPS 2026 · similarity 0.85)
- Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.