Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Bandits with Abstention under Expert Advice
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CBA, a bandit algorithm for expert advice where one action is abstention (zero reward), giving expected-reward bounds for confidence-rated experts and applying it to adversarial contextual bandits with abstention.
Problem
Prediction with expert advice under bandit feedback when the learner can abstain (a zero-reward action), including confidence-rated predictors and adversarial contextual bandits where abstaining is allowed; prior bounds only hold in restricted cases or treat abstention as a regular action.
Approach
Modified mirror descent with unnormalized relative entropy, projecting weights onto the set of valid stochastic policies (confidence-weighted sums ≤1) and using an Exp3/Exp4-style unbiased reward estimator; this yields expected cumulative reward bounds for general confidence-rated experts, and with ball/graph bases on contexts gives per-trial runtime near-linear in the number of contexts.
Key results
- Theorem gives expected cumulative reward lower bound of Exp4-type order, improving on SpecialistExp for specialists; per-trial time O(KE).
- Contextual bound sqrt(2M ln(N)(6K+1)T) with per-trial time O(KN) for N basis elements; ball-based implementation reduces time to O(KN ln N).
- Experiments on graph node classification show the method outperforms ContextualBandit and GABA-II on synthetic graphs over 20 runs, with d_inf balls performing particularly well; on Cora it matches GABA-II except with community basis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CBA instead of Exp4 when comparing against general valid expert combinations | Bandit prediction with expert advice under abstention · General confidence-rated experts | expected cumulative reward bound: 12K+2; Exp4 replaces this term with 8K+8 and requires \|\bs{u}\|_1=1 | Helped |
| Use CBA with a basis matched to graph structure | Contextual bandits with abstention · Stochastic block model and Gaussian graph | performance | Helped |
| Use d-infinity balls as the CBA basis | CBA with graph bases · Stochastic block model and Gaussian graph | performance | Helped |
| Use the community detection (LVC) basis rather than other tested bases | CBA with graph bases · Cora | performance | Mixed |
| Use the LVC basis | CBA with graph bases · LastFM Asia | performance | Helped |
| Choose a basis that closely matches the context structure | CBA with graph bases · Graph experiments | performance | Mixed |
| Use the tree-based ball implementation instead of the direct implementation | CBA for metric-space contexts · Metric balls | per-trial time complexity: \mathcal{O}(KN^2) -> \mathcal{O}(KN\ln(N)) | Helped |
Limitations
- Experiments are preliminary and limited to graph node classification with synthetic noise injection
- Adversary must be fully oblivious; results require finite context sets
- Per-trial runtime of direct implementation is O(KN^2) for generic bases
Experimental setup
Datasets: Stochastic block model (synthetic) Gaussian graph (synthetic) Cora LastFM Asia
Benchmarks: Stochastic block model Gaussian graph Cora LastFM Asia
Metrics: number of mistakes over time cumulative reward
Baselines: ContextualBandit with similarity GABA-II EXP3 per context
Builds on: Exp4 SpecialistExp Exp3 GABA
Closest papers by meaning
- On the Minimax Regret for Contextual Linear Bandits and Multi-Armed Bandits with Expert Advice (NeurIPS 2024 · similarity 0.89)
- Regret Bounds for Adversarial Contextual Bandits with General Function Approximation and Delayed Feedback (NeurIPS 2025 · similarity 0.88)
- Sparsity-Agnostic Linear Bandits with Adaptive Adversaries (NeurIPS 2024 · similarity 0.88)
- Causal Bandits: The Pareto Optimal Frontier of Adaptivity, a Reduction to Linear Bandits, and Limitations around Unknown Marginals (ICML 2024 · similarity 0.88)
- Bayesian Regret Minimization in Offline Bandits (ICML 2024 · similarity 0.88)
- Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.