Research map / Label-efficient and robust learning / Recognition and long-tail
Uncertainty Herding: One Active Learning Method for All Label Budgets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Uncertainty Herding, a greedy maximization of an uncertainty-weighted generalized coverage objective with temperature-scaling and radius adaptation, to make active learning work across both low and high label budgets.
Problem
Active learning methods either excel at high label budgets (uncertainty-based) or low budgets (representation-based), and existing budget-switching methods are unreliable, so a single method is needed that works across label budgets.
Approach
Introduce uncertainty coverage, which weights generalized (kernel) coverage by per-point uncertainty, and greedily maximize its estimate (Uncertainty Herding). Parameter adaptation makes temperature grow with few labels (near-constant uncertainty, so it reduces to MaxHerding) and shrinks the kernel radius as labels grow (so it reduces to uncertainty sampling).
Key results
- Averaged over budget regimes, UHerding gains over Random are positive in every regime (e.g., low-budget CIFAR-10 +5.5, CIFAR-100 +5.5; high-budget CIFAR-10 +3.0), exceeding every competitor in the summary table.
- On DomainNet with DeiT Base, UHerding beats MaxHerding by 1.5-2% even at low budgets and beats ALFA-Mix by up to 13% at low budgets.
- Ablation on CIFAR-10 shows temperature scaling and radius adaptation both help, and together they outperform both MaxHerding and Margin across all budget regimes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UHerding instead of MaxHerding | active learning · CIFAR-10, CIFAR-100, TinyImageNet, DomainNet, and ImageNet; low and middle budgets | ΔAcc over Random | Helped |
| Use UHerding instead of Margin | active learning · low-budget tasks | accuracy | Helped |
| Use UHerding instead of representation-based methods | active learning · high-budget tasks | accuracy | Helped |
| Use temperature scaling for the uncertainty function (ablation) | UHerding · CIFAR-10, all budget regimes | ΔAcc | Helped |
| Adapt the coverage radius (ablation) | UHerding · CIFAR-10, high-budget regime | ΔAcc | Helped |
| Use radius adaptation without temperature scaling (ablation) | UHerding · CIFAR-10, across budget regimes | ΔAcc | Mixed |
| Combine temperature scaling and radius adaptation (ablation) | UHerding · CIFAR-10, all budget regimes | ΔAcc | Helped |
| Use fixed parameters instead of parameter adaptation (ablation) | UHerding · low- and mid-budget regimes | accuracy | Hurt |
| Use fixed parameters instead of parameter adaptation (ablation) | UHerding · high-budget regime | accuracy | Hurt |
| Replace k-means++ with MaxHerding (ablation) | BADGE · CIFAR-100 | accuracy | Helped |
| Add parameter adaptation (ablation) | BADGE-derived method · CIFAR-100 | accuracy | Helped |
| Change the uncertainty choice to Margin (ablation) | BADGE-derived method · CIFAR-100 | accuracy | Helped |
Surprising findings
- Choice of uncertainty measure (Margin, Entropy, Confidence) has little impact on UHerding performance.
- ActiveFT consistently underperforms the other methods, possibly because it is designed for single-iteration selection.
Limitations
- Budget regimes are manually categorized as low, mid, or high for each dataset rather than determined automatically.
- Parameter adaptation relies on heuristics (min labeled-pair distance for radius, ECE-based temperature selection) without theoretical justification for their optimality.
- Theoretical guarantees rely on approximate greedy optimization with a (1-1/e) factor and an estimation error term that is not tight for the experimental settings.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet DomainNet ImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet DomainNet ImageNet
Metrics: accuracy improvement over Random selection (mean delta Acc a
Baselines: Random Entropy Margin Confidence BADGE ALFA-Mix Weighted k-means ActiveFT Typiclust Coreset MaxHerding
Builds on: MaxHerding generalized coverage BADGE ALFA-Mix ActiveFT
Closest papers by meaning
- Uncertainty for Active Learning on Graphs (ICML 2024 · similarity 0.88)
- Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.87)
- Adaptive Labeling for Efficient Out-of-distribution Model Evaluation (NeurIPS 2024 · similarity 0.86)
- Robust Sampling for Active Statistical Inference (NeurIPS 2025 · similarity 0.86)
- Performative Prediction with Selective Labels (NeurIPS 2026 · similarity 0.86)
- Nearly Optimal Sample Complexity for Learning with Label Proportions (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.