Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
One-shot Active Learning Based on Lewis Weight Sampling for Multiple Deep Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a one-shot active learning method for multiple deep target models that samples unlabeled instances by their maximum Lewis weights across network backbones' representations and solves l_p regression on each model's linear head, with theoretical sample-complexity bounds.
Problem
Active learning for multiple deep target models is costly because iterative methods retrain all models at every query round; existing one-shot methods rely on feature distances that ignore the distinct representations of multiple networks.
Approach
Treat each deep model as a fixed backbone plus a linear prediction layer, and select a single shared reweighted sampling matrix by sampling unlabeled rows with probability proportional to the maximum Lewis weight across the models' feature matrices, which yields l_p-regression guarantees simultaneously for all models in one shot.
Key results
- In the vanilla deep learning scenario, the one-shot method achieves performance comparable to iterative AL methods (DIAM, QBC, Entropy) in most cases and outperforms the other one-shot baselines Coreset and Random-style methods on the classification benchmarks.
- Running time on classification with 15000 queries: the proposed method takes about 5.3 GPU hours on MNIST versus 46.6 for DIAM and 23.9 for QBC.
- The sum of maximum leverage scores across the 50 representations grows very slowly as the number of models increases, indicating strongly correlated leverage scores across deep representations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use one-shot maximum Lewis weight sampling instead of the compared baselines | Multiple deep models, vanilla learning · Classification benchmarks | mean accuracy | No clear effect |
| Use one-shot maximum Lewis weight sampling instead of the compared baselines | Multiple deep models, fine-tuning · Biwi, FLD, CelebA | mean MSE | Helped |
| Use one-shot maximum Lewis weight sampling instead of iterative active learning | Multiple deep models · Classification benchmarks | running time | Helped |
| Use maximum Lewis weight sampling rather than Random sampling | Multiple deep models · Biwi | running time (hours): 1.217 -> 1.551 | Hurt |
| Use maximum Lewis weight sampling rather than Random sampling | Multiple deep models · FLD | running time (hours): 1.383 -> 1.850 | Hurt |
| Use maximum Lewis weight sampling rather than Random sampling | Multiple deep models · CelebA | running time (hours): 1.202 -> 5.817 | Hurt |
| Use maximum Lewis weight sampling rather than Random sampling | Multiple deep models · MNIST | class imbalance ratio: 1.262 -> 1.217 | Helped |
| Use maximum Lewis weight sampling rather than Random sampling | Multiple deep models · Fashion-MNIST | class imbalance ratio: 1.091 -> 1.081 | Helped |
| Use maximum Lewis weight sampling rather than Random sampling | Multiple deep models · CIFAR-100 | class imbalance ratio: 1.511 -> 1.440 | Helped |
| Use Coreset selection instead of Random sampling | Multiple deep models, vanilla learning · Classification benchmarks | mean accuracy | Hurt |
| Use Entropy selection instead of Random sampling | Multiple deep models, vanilla learning · Classification benchmarks | mean accuracy | No clear effect |
| Use DIAM instead of one-shot maximum Lewis weight sampling | Multiple deep models, vanilla learning · Classification benchmarks | mean accuracy | Mixed |
Surprising findings
- Coreset is usually worse than random sampling when many distinct target models are trained, since selected instances are tailored to the supernet representation.
- DIAM is effective on MNIST-family datasets but unstable and fails on other datasets.
Limitations
- Theoretical guarantees assume a fixed backbone and a single linear prediction layer per model, which only approximates full network training.
- Sample complexity bound is proportional to the sum of maximum Lewis weights, which could in the worst case be as large as kd.
- Vanilla deep learning experiments depend on a specific NAS model family (OFA sub-networks), limiting the generality of the empirical findings.
Experimental setup
Datasets: MNIST Fashion-MNIST Kuzushiji-MNIST SVHN EMNIST-digits EMNIST-letters CIFAR-10 CIFAR-100 Biwi FLD CelebA
Benchmarks: MNIST Fashion-MNIST Kuzushiji-MNIST SVHN EMNIST-digits EMNIST-letters CIFAR-10 CIFAR-100 Biwi FLD CelebA
Metrics: mean accuracy mean MSE running time class imbalance ratio
Baselines: DIAM QBC Entropy Coreset Random
Builds on: DIAM Gajjar et al. single-neuron active learning Lewis weight sampling OFA
Closest papers by meaning
- Active Learning for Multiple Target Models (NeurIPS 2022 · similarity 0.90)
- Batch Active Learning at Scale (NeurIPS 2021 · similarity 0.87)
- Composite Active Learning: Towards Multi-Domain Active Learning with Theoretical Guarantees (AAAI 2024 · similarity 0.86)
- Active Learning Guided by Efficient Surrogate Learners (AAAI 2024 · similarity 0.85)
- AutoAL: Automated Active Learning with Differentiable Query Strategy Search (ICML 2025 · similarity 0.85)
- Querying Easily Flip-flopped Samples for Deep Active Learning (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.