Research map / Label-efficient and robust learning / OOD and anomaly detection

ASAT: Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection

NeurIPS 2026 · Label-efficient and robust learning › OOD and anomaly detection · approach: Provable and feature-based OOD detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ASAT, a human-in-the-loop OOD detector that jointly adapts scoring functions and thresholds from importance-sampled human feedback to maximize TPR while controlling FPR over deployment.

Problem

OOD detectors set thresholds using only ID data for a target TPR, which leaves FPR uncontrolled (often 32-91% on OpenOOD CIFAR-10), and fixed scores cannot adapt to evolving OOD inputs.

Approach

Jointly update the scoring function (via a smooth surrogate of TPR/FPR with a penalty trade-off) and the threshold (via a UCB-constrained binary search on importance-sampled FPR estimates), with a DKW-based model selection check, giving time-uniform FPR control via martingale LIL bounds.

Key results

• In stationary settings ASAT keeps FPR below 5% across four settings, while FSFT shows very high FPR at TPR-95%.
• With EBO as the starting score, ASAT shows over a 40% TPR improvement over FSAT toward the end of deployment.
• Under nonstationary shift, FPR violations occur only when the new OOD is less distinguishable from ID than the old OOD; ASAT adapts with transient violations.

Surprising findings

• The theoretical UCB is too pessimistic, so experiments use a heuristic UCB with tuned constants; the all-history estimator without sample splitting is used in practice, outside the analyzed calibration-safe variant.

Limitations

• Assumes expert (noise-free) human labels.
• Guarantees assume stationary OOD distributions; nonstationary analysis left open.
• Experiments use a heuristic UCB and no sample splitting, so the theory does not directly cover the reported experiments; only semantic shifts are studied.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet Places365 MNIST SVHN Texture ImageNet-1k

Benchmarks: OpenOOD

Metrics: TPR at FPR 5% FPR TPR

Baselines: FSFT FSAT naive binary classifier

Builds on: FSAT OpenOOD ODIN Energy score Martingale LIL bound

Closest papers by meaning

• Leveraging Perturbation Robustness to Enhance Out-of-Distribution Detection (CVPR 2025 · similarity 0.86)
• DOS: Diverse Outlier Sampling for Out-of-Distribution Detection (ICLR 2024 · similarity 0.85)
• Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.85)
• ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection (ICLR 2024 · similarity 0.85)
• Discriminability-Driven Channel Selection for Out-of-Distribution Detection (CVPR 2024 · similarity 0.85)
• Enhancing the Power of OOD Detection via Sample-Aware Model Selection (CVPR 2024 · similarity 0.85)