Research map / Label-efficient and robust learning / Recognition and long-tail
SEL-BALD: Deep Bayesian Active Learning with Selective Labels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies active learning when humans can reject labeling requests (selective labels), and proposes SEL-BALD acquisition functions (e-BALD, Joint-BALD, Joint-BALD-UCB/TS) that jointly target model information gain and human discretion-model information gain.
Problem
Active learning in high-stakes settings where human experts may decline to label instances selected by the model, so labels are selectively observed and conventional acquisition wastes examination budget.
Approach
Weights the BALD mutual-information objective by the estimated probability that a human will label an instance, and adds an information-gain term about a Bayesian human-discretion model, with UCB or Thompson-sampling variants to balance exploration and exploitation of the discretion model.
Key results
- On the synthetic two-circle data, Joint-BALD-UCB and Joint-BALD-TS reach accuracy 0.99 and 0.98 versus 0.78 for RANDOM and 0.55 for Naive-BALD at budget 450.
- On GMC and MNIST, Joint-BALD-UCB and Joint-BALD-TS outperform Naive-BALD and RANDOM across budgets, which spend most of the budget on examinations that are rejected.
- Under homogeneous discretion (e(x)=0.8 on MNIST), all SEL-BALD variants perform similarly, consistent with the theory that ALIR reduces to standard active learning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Joint-BALD-UCB instead of Naive-BALD | SEL-BALD active learning · Synthetic data | accuracy: 0.55 -> 0.99 | Helped |
| Use Joint-BALD-TS instead of Naive-BALD | SEL-BALD active learning · Synthetic data | accuracy: 0.55 -> 0.98 | Helped |
| Model human discretion and explore its uncertainty with Joint-BALD-UCB | SEL-BALD active learning · Give-Me-some-Credit (GMC) | accuracy | Helped |
| Model human discretion and explore its uncertainty with Joint-BALD-TS | SEL-BALD active learning · MNIST | accuracy | Helped |
| Use Naive-BALD rather than Joint-BALD-UCB | Active learning with instance rejection · Synthetic data | accuracy: 0.99 -> 0.55 | Hurt |
| Use RANDOM rather than Joint-BALD-UCB | Active learning with instance rejection · Synthetic data | accuracy: 0.99 -> 0.78 | Hurt |
| Use e-BALD rather than Joint-BALD-UCB | SEL-BALD active learning · Synthetic data | accuracy: 0.99 -> 0.76 | Hurt |
| Use e-BALD with a noisy human-discretion estimate instead of Joint-BALD-UCB | SEL-BALD active learning · Synthetic data | accuracy: 0.99 -> 0.76 | Hurt |
| Use Joint-BALD rather than Joint-BALD-UCB | SEL-BALD active learning · Synthetic data | accuracy: 0.99 -> 0.77 | Hurt |
| Use BALD rather than Joint-BALD-UCB under changing human behavior | Active learning with instance rejection · Synthetic two-moons data | accuracy | Hurt |
| Use Joint-BALD-UCB rather than Naive-BALD with a logistic-regression human behavior model | SEL-BALD active learning · Adult | accuracy | Helped |
| Replace information gain with entropy and use Joint-Entropy-UCB (ablation) | Entropy-based active learning · Synthetic data | accuracy | Helped |
Surprising findings
- e-BALD, which greedily targets likely-labeled instances, can underperform Naive-BALD-like baselines at larger budgets because of noisy discretion-model estimates.
- RANDOM is a strong baseline on the Mushroom dataset in the batch setting, per the ML-based human behavior appendix.
Limitations
- Assumes all humans share a single discretion model rather than heterogeneous decision makers.
- Performance depends on correct specification of the human discretion model class; misspecification can bias acquisition.
- Fairness implications of prioritizing labeled instances are not addressed and the method may not apply in some settings.
Experimental setup
Datasets: Give-Me-Some-Credit MNIST Fashion-MNIST CIFAR-10 Adult Mushroom synthetic two-circle synthetic two-moon
Benchmarks: Give-Me-Some-Credit MNIST Fashion-MNIST CIFAR-10 Adult Mushroom synthetic two-circle
Metrics: accuracy number of labeled samples examination and labeling budget fraction
Baselines: RANDOM Naive-BALD e-BALD Joint-BALD
Builds on: BALD MC dropout UCB Thompson sampling
Closest papers by meaning
- Instance-wise Supervision-level Optimization in Active Learning (CVPR 2025 · similarity 0.85)
- Deep Bayesian Active Learning for Preference Modeling in Large Language Models (NeurIPS 2024 · similarity 0.84)
- Uncertainty Herding: One Active Learning Method for All Label Budgets (ICLR 2025 · similarity 0.84)
- Regretful Decisions under Label Noise (ICLR 2025 · similarity 0.84)
- Contextual Active Model Selection (NeurIPS 2024 · similarity 0.84)
- Adaptive Labeling for Efficient Out-of-distribution Model Evaluation (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.