Research map / Label-efficient and robust learning / Recognition and long-tail
Instance-wise Supervision-level Optimization in Active Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ISO, an active learning framework that jointly selects instances and their supervision level (full exact labels vs. weak superclass labels) under a fixed budget using value-to-cost ratios and a diversity-aware k-means++-style batch selection.
Problem
Active learning under a fixed annotation budget where instances can be annotated either with expensive exact labels or cheaper superclass (weak) labels; the optimal supervision level per instance is unknown.
Approach
Scores each unlabeled instance under both full and weak supervision by a value-to-cost ratio (expected model improvement times percentile-normalized margin uncertainty, divided by annotation cost), then selects a diverse batch by sampling vectors (VCR-scaled normalized features) with k-means++-style seeding until the budget is exhausted, letting full and weak labels compete per instance.
Key results
- On CIFAR-100 at round 5, ISO achieves over 10% higher accuracy than conventional full-supervision AL methods, and reaches ~30% accuracy in three rounds where baselines need five.
- ISO outperforms APFWA and all conventional AL baselines on CIFAR-100 and CUB-200 across all rounds with weak-supervision cost 1/2.
- Cheaper weak supervision (Cw = 1/4, 1/8) further improves ISO accuracy, and ISO beats fixed full/weak proportion baselines (F(x%)+W(y%)) without needing to tune the proportion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ISO instead of conventional full-supervision active learning methods | CIFAR100 · CIFAR100 | accuracy: over 10% higher accuracy | Helped |
| Use ISO instead of conventional full-supervision active learning methods | CIFAR100 · CIFAR100 | annotation budget to reach approximately 30% accuracy: three-fifths of the budget | Helped |
| Use ISO instead of APFWA | CIFAR100 · CIFAR100 | accuracy | Helped |
| Use ISO instead of conventional active learning methods | ISO · CUB200 | accuracy | Helped |
| Decrease weak-supervision cost from 1/2 to 1/4 or 1/8 | ISO · CIFAR100 | accuracy: Cw = 1/2, 1/4, and 1/8 | Helped |
| Decrease weak-supervision cost from 1/2 to 1/4 or 1/8 | ISO · CUB200 | accuracy: Cw = 1/2, 1/4, and 1/8 | Helped |
| Dynamically optimize supervision levels and instance selection instead of fixing the full/weak supervision proportions | ISO · CIFAR100 | accuracy | Helped |
| Remove instance-wise uncertainty from VCR calculation (ablation) | ISO · CIFAR100 | accuracy | Hurt |
| Remove instance-wise uncertainty from VCR calculation (ablation) | ISO · CUB200 | accuracy | Hurt |
| Remove diversity-aware batch selection and greedily select instances by VCR (ablation) | ISO · CIFAR100 | accuracy | Hurt |
| Remove diversity-aware batch selection and greedily select instances by VCR (ablation) | ISO · CUB200 | accuracy | Hurt |
Limitations
- Validated only on classification tasks with CIFAR-100 and CUB-200.
- Assumes exactly two supervision levels (full and superclass).
- Uses a fixed weak-supervision cost defined in advance rather than learned.
Experimental setup
Datasets: CIFAR-100 CUB-200-2011
Benchmarks: CIFAR-100 CUB-200-2011
Metrics: classification accuracy
Baselines: Random Margin MaxConf Entropy Coreset ALBL BADGE APFWA
Builds on: BADGE APFWA k-DPP Margin uncertainty sampling
Closest papers by meaning
- Re-Active Learning: Active Learning with Relabeling (AAAI 2016 · similarity 0.88)
- Towards Cost-Effective Learning: A Synergy of Semi-Supervised and Active Learning (CVPR 2025 · similarity 0.88)
- Cost-Accuracy Aware Adaptive Labeling for Active Learning (AAAI 2020 · similarity 0.88)
- Improved Algorithm for Deep Active Learning under Imbalance via Optimal Separation (ICML 2025 · similarity 0.87)
- A Cross-Domain Benchmark for Active Learning (NeurIPS 2024 · similarity 0.86)
- Contextual Active Model Selection (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.