Research map / Bandits and online learning / Contextual bandits and Thompson sampling

Neural Active Learning Beyond Bandits

ICLR 2024 · Bandits and online learning › Contextual bandits and Thompson sampling · approach: Causal and structured bandits · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Extends bandit-based neural active learning to multi-class classification with a neural exploitation/exploration design, giving stream-based (NeurONAL-Stream) and pool-based (NeurONAL-Pool) algorithms with regret guarantees.

Problem

Active learning with neural networks in stream-based and pool-based settings, where prior bandit-based transformations scale poorly with the number of classes K and input dimension.

Approach

Replaces the per-class bandit arm transformation with neural exploitation and exploration networks that output K-class probabilities jointly from the instance, uses an end-to-end embedding for exploration, and for pool-based selection applies an inverse-gap weighting sampling rule over the pool, with regret bounds that drop the R complexity term.

Key results

• NeurONAL-Pool reaches 83.9% testing accuracy on Letter and 66.1% on Covertype, above CoreSet (80.4% and 62.0%) under the rebuttal setup.
• NeurONAL-Pool runs in 13.14 s on MT versus 32.56 s for CoreSet and 548.79 s for BADGE in the pool-based running-time table.
• The regret bound's sqrt(K) dependence can be removed under a 0-1 loss or when ||h(x)||_2 <= 1, and the tighter analysis matches minimax active learning rates in the beta=0 and beta to infinity cases.

Measured effects reported in the paper

Surprising findings

• Uniformly feeding pool data to the stream-based learner (Neu-UniS) performs worse than the dedicated pool-based method, suggesting stream-based algorithms cannot simply be converted to the pool setting.

Limitations

• Regret bound depends on the complexity term S*, which is argued to be O~(1) only under data non-degeneracy assumptions and wide networks.
• The regret bound still carries a sqrt(K) factor for general bounded losses.
• ALPS performance depends on pretrained model pool and gives poor results on Fashion.

Experimental setup

Datasets: Adult Covertype Fashion-MNIST MT Letter Shuttle

Benchmarks: Adult Covertype Fashion-MNIST MT Letter Shuttle

Metrics: testing accuracy running time cumulative regret

Baselines: CoreSet BADGE DynamicAL NeuAL-NTK I-NeurAL ALPS

Builds on: NeuAL NeuralUCB Neural bandits (Zhou et al. 2020) Ban et al. 2022 neural active learning bandit

Closest papers by meaning

• Improved Algorithms for Neural Active Learning (NeurIPS 2022 · similarity 0.91)
• Towards Robust and Reproducible Active Learning Using Neural Networks (CVPR 2022 · similarity 0.89)
• Budgeted stream-based active learning via adaptive submodular maximization (NeurIPS 2016 · similarity 0.88)
• Neural Active Learning with Performance Guarantees (NeurIPS 2021 · similarity 0.88)
• Transductive Active Learning: Theory and Applications (NeurIPS 2024 · similarity 0.88)
• Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples (ICML 2024 · similarity 0.87)