Research map / Label-efficient and robust learning / Domain adaptation and generalization
Towards Domain Adaptive Neural Contextual Bandits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DABand, a domain-adaptive neural contextual bandit that explores a high-cost target domain by collecting feedback only from a low-cost source domain, using a regret bound and adversarial representation alignment.
Problem
Adapting contextual bandit policies from a low-cost source domain with reward feedback to a high-cost target domain where no reward feedback can be collected, under distribution shift (zero-shot target regret).
Approach
Derives a target-regret upper bound for cross-domain bandits (source regret, regression error, H-Delta-H divergence for regression, predicted-reward regularizer) and turns it into a training objective combining Neural-LinUCB source updates with a regression loss, reward regularizer, and adversarial domain discriminator for encoder alignment.
Key results
- On DIGIT, DABand reaches 0.6002 average zero-shot target accuracy versus 0.3816 for NLinUCB and 0.3808 for LinUCB.
- On VisDA17, DABand reaches 0.4644 average accuracy versus 0.1001 for NLinUCB and 0.1145 for LinUCB-P.
- On S2RDA49, DABand reaches 0.3923 average accuracy versus 0.1108 for NLinUCB; the ablation removing both regression and predicted-reward terms drops DIGIT to 0.5676.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DABand instead of NLinUCB | zero-shot target contextual bandits · DIGIT target | accuracy: +0.2186 | Helped |
| Use DABand instead of NLinUCB | zero-shot target contextual bandits · VisDA17 target | accuracy: +0.3643 | Helped |
| Use DABand instead of NLinUCB | zero-shot target contextual bandits · S2RDA49 target | accuracy: +0.2815 | Helped |
| Add regression error term (ablation) | DABand · DIGIT | accuracy: 0.5682 -> 0.6002 | Helped |
| Add regression error term (ablation) | DABand · VisDA17 | accuracy: 0.4096 -> 0.4644 | Helped |
| Add regression error term (ablation) | DABand · S2RDA49 | accuracy: 0.3694 -> 0.3923 | Helped |
| Add predicted reward term (ablation) | DABand · DIGIT | accuracy: 0.5768 -> 0.6002 | Helped |
| Add predicted reward term (ablation) | DABand · VisDA17 | accuracy: 0.4304 -> 0.4644 | Helped |
| Add predicted reward term (ablation) | DABand · S2RDA49 | accuracy: 0.3719 -> 0.3923 | Helped |
| Add regression error and predicted reward terms (ablation) | DABand · DIGIT | accuracy: 0.5676 -> 0.6002 | Helped |
| Add regression error and predicted reward terms (ablation) | DABand · VisDA17 | accuracy: 0.4088 -> 0.4644 | Helped |
| Add regression error and predicted reward terms (ablation) | DABand · S2RDA49 | accuracy: 0.3691 -> 0.3923 | Helped |
Surprising findings
- Plain neural contextual bandits (NLinUCB) and LinUCB perform worse than or comparable to a simple PCA variant on some target-domain settings, suggesting that representation learning amplifies source overfitting under domain shift.
- NLinUCB-P achieves the highest per-class accuracy on some DIGIT classes (0.8588 on class 0) but near-zero accuracy on others, indicating unstable per-class behavior.
Limitations
- Theory relies on the H-Delta-H divergence and bounded-reward assumptions; the bound includes a constant term that is claimed negligible without full justification.
- Evaluated only on image classification-derived bandit settings (DIGIT, VisDA17, S2RDA49) rather than real drug-response or user-interaction data.
- Alignment is only as good as the adversarial discriminator; the paper notes that aligning the rich target domain with a sparse source domain remains open.
Experimental setup
Datasets: DIGIT VisDA17 S2RDA49
Benchmarks: DIGIT VisDA17 S2RDA49
Metrics: zero-shot target accuracy (1 - R_T/N) per-class accuracy cumulative regret
Baselines: LinUCB LinUCB-P NLinUCB NLinUCB-P
Builds on: LinUCB Neural-LinUCB domain-adversarial training
Closest papers by meaning
- Learning Neural Contextual Bandits through Perturbed Rewards (ICLR 2022 · similarity 0.89)
- Neural Dueling Bandits: Preference-Based Optimization with Human Feedback (ICLR 2025 · similarity 0.88)
- Efficient Contextual Bandits with Uninformed Feedback Graphs (ICML 2024 · similarity 0.88)
- Robust Neural Contextual Bandit against Adversarial Corruptions (NeurIPS 2024 · similarity 0.87)
- Diffusion Models Meet Contextual Bandits (NeurIPS 2025 · similarity 0.87)
- Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedback (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.