Research map / Label-efficient and robust learning / OOD and anomaly detection

Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning

NeurIPS 2026 · Label-efficient and robust learning › OOD and anomaly detection · approach: Uncertainty and misclassification detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes evaluating epistemic uncertainty by its ability to identify regret (reducible error) via a unified constrained selective-prediction framework, and benchmarks UQ methods on dense-annotation datasets, showing rank inversions versus OOD/active-learning proxies.

Problem

Epistemic uncertainty is evaluated with proxy tasks (OOD detection, active learning) whose Bayes-optimal scorers differ from the regret-minimizing selector, and rank correlation between learned aleatoric and epistemic components is used as a disentanglement criterion, which may not reflect operation

Approach

Unifies selective classification and the epistemic reject-option into one constrained optimization over coverage, risk and regret, showing the optimal selector thresholds a convex combination of aleatoric and epistemic uncertainty; proposes Pareto-gap (IGD+ distance to the oracle risk-regret-coverage surface) as a joint disentanglement diagnostic, with AuReC for pure regret ranking.

Key results

• On CIFAR-10H (CE), Evidential networks achieve the lowest AuReC (0.0673) and lowest P-gap (0.0379) despite r_s(Â,Ê)=0.999.
• DDU ranks best on Far-OOD AuROC (0.935) on CIFAR-10 but worst in regret on CIFAR-10H (AuReC 0.0958).
• Evidential networks reach only 0.177 NAUC on OpenML 6 active learning despite strong regret performance on CIFAR-10H.

Surprising findings

• Evidential networks have near-perfect learned-component rank correlation (0.999) yet the best Pareto-gap on CIFAR-10H, undermining correlation-based disentanglement criteria.
• Method rankings under OOD detection and active learning invert relative to regret-based rankings.

Limitations

• Exact regret requires dense human annotations p*(y|x), limiting applicability to standard single-label settings.
• OOD and active learning results are adopted from a concurrent submission on different datasets, not evaluated on the same data.
• Oracle linear-combination search on the test set makes the Pareto-gap an upper bound rather than a deployable method.

Experimental setup

Datasets: CIFAR-10 CIFAR-10H DCIC suite (8 datasets) APPA-REAL OpenOOD OpenML

Benchmarks: CIFAR-10H DCIC suite APPA-REAL OpenOOD OpenML tabular active learning

Metrics: AuReC ex-AuReC n-AuReC AuRC Pareto-gap AuROC NAUC Spearman rank correlation

Baselines: MC-Dropout Deep Ensembles DDU Evidential networks

Builds on: epistemic reject-option selective classification deep ensembles MC-dropout DDU

Closest papers by meaning

• Rethinking Aleatoric and Epistemic Uncertainty (ICML 2025 · similarity 0.90)
• Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks (NeurIPS 2024 · similarity 0.88)
• Epistemic Uncertainty Quantification For Pre-Trained Neural Networks (CVPR 2024 · similarity 0.87)
• Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods? (ICML 2024 · similarity 0.87)
• Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? (NeurIPS 2024 · similarity 0.86)
• Epistemic Uncertainty for Generated Image Detection (NeurIPS 2025 · similarity 0.85)