Research map / Label-efficient and robust learning / Recognition and long-tail
Rethinking Epistemic and Aleatoric Uncertainty for Active Open-Set Annotation: An Energy-Based Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EAOA, an energy-based active open-set annotation framework that first filters candidates by low energy-based epistemic uncertainty and then queries high energy-based aleatoric uncertainty examples with an adaptive candidate-set size.
Problem
Active learning with open-set (unknown) classes in unlabeled pools, where existing methods either query low-epistemic-uncertainty examples (low information) or high-aleatoric-uncertainty examples (AU is meaningless for unknown-class examples).
Approach
Combine epistemic and aleatoric uncertainty via free energy: a (C+1)-class detector gives an energy-based EU score (learning-based plus data-driven, with a margin-based energy loss), and a target classifier gives an energy-based AU score; a coarse-to-fine sampler first keeps the k*b lowest-EU examples, then picks the b highest-AU ones, with k adapted by target query precision.
Key results
- Across CIFAR-10, CIFAR-100 and Tiny-ImageNet with mismatch ratios 20-40% (10-20% for Tiny-ImageNet), EAOA achieves the best final-round test accuracy in most settings compared with Random, Uncertainty, Certainty, Coreset, BADGE, CCAL, MQNet, EOAL and BUAL.
- EAOA maintains high query precision and has the shortest runtime among the AOSA methods on CIFAR-100 at 40% mismatch ratio, according to the paper.
- Ablation on CIFAR-100 (40% mismatch) shows energy-based EU alone yields large gains, and the energy-based AU alone performs poorly but improves substantially when combined with EU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EAOA instead of the compared active open-set annotation methods | EAOA · CIFAR-10, CIFAR-100, and Tiny-ImageNet | test accuracy | Helped |
| Use EAOA instead of the compared active open-set annotation methods | EAOA · CIFAR-10, CIFAR-100, and Tiny-ImageNet | mean query precision | Helped |
| Use energy-based epistemic uncertainty with energy-based aleatoric uncertainty, rather than either alone (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | test performance | Helped |
| Use energy-based epistemic uncertainty with energy-based aleatoric uncertainty, rather than either alone (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | mean query precision | Helped |
| Add energy-based epistemic uncertainty that uses labeled unknown-class examples, rather than using free energy alone (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | test performance | Helped |
| Add energy-based aleatoric uncertainty without energy-based epistemic uncertainty (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | test performance | Hurt |
| Add data-driven epistemic uncertainty (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | test performance | Helped |
| Add margin-based energy loss to detector training (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | test performance | Helped |
| Increase target query precision tP (ablation) | EAOA · CIFAR-10 and CIFAR-100 | query recognition performance | Helped |
| Vary target query precision tP (ablation) | EAOA · CIFAR-10 and CIFAR-100 | test performance | No clear effect |
| Increase energy loss weight ωe (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | detector accuracy | Hurt |
| Decrease energy loss weight ωe (ablation) | EAOA · CIFAR-100, 40% mismatch ratio | detector recognition performance | Hurt |
Surprising findings
- Entropy-based certainty sampling performs similarly to Random, supporting that aleatoric uncertainty is only meaningful in closed-set settings.
- Energy-based aleatoric uncertainty alone performs poorly, but combined with the epistemic filter it gives significant improvement.
Limitations
- Adds several hyperparameters (K, tP, k1, a, z, margins, energy-loss weight) although the authors say they are easier to set.
- Evaluated only on image classification benchmarks with ResNet-18 and small-scale open-set splits.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: test accuracy query precision mean query precision runtime
Baselines: Random Uncertainty Certainty Coreset BADGE CCAL MQNet EOAL BUAL LfOSA
Builds on: LfOSA EOAL BUAL energy-based models
Closest papers by meaning
- Revisiting Unknowns: Towards Effective and Efficient Open-Set Active Learning (CVPR 2026 · similarity 0.90)
- Entropic Open-Set Active Learning (AAAI 2024 · similarity 0.88)
- Inconsistency-Based Data-Centric Active Open-Set Annotation (AAAI 2024 · similarity 0.88)
- Active Learning for Open-Set Annotation (CVPR 2022 · similarity 0.87)
- Learn More with Less: Uncertainty Consistency Guided Query Selection for RLVR (ICLR 2026 · similarity 0.84)
- Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.