Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Leveraging Offline Data in Linear Latent Contextual Bandits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SOLD to estimate a low-rank reward subspace from unlabeled offline latent-bandit trajectories, then two online algorithms (LOCAL-UCB, minimax optimal, and the computationally efficient ProBALL-UCB) that use it to accelerate LinUCB, plus a de Finetti theorem showing exchangeable coherent stateless decision processes are latent bandits.
Problem
Using unlabeled offline bandit trajectories from heterogeneous users with latent states to accelerate online linear contextual bandit learning when per-user reward parameters lie in a low-rank latent subspace of unknown dimension.
Approach
Estimate the reward-parameter subspace offline via split-trajectory outer products with regularization corrections (SOLD), then optimistically use it online: LOCAL-UCB optimizes over the intersection of offline subspace-restricted and full-dimensional confidence sets, while ProBALL-UCB switches between a projected LinUCB and standard LinUCB using a computable proxy condition.
Key results
- LOCAL-UCB achieves regret O(min(d_A sqrt T, d_K sqrt T (1+ sqrt(d_A T/(d_K N))))) and a matching lower bound shows it is minimax optimal.
- ProBALL-UCB, initialized with SOLD, performs no worse than LinUCB and outperforms mUCB, mmUCB, TS, mmTS and MixTS in simulation and on MovieLens (d_K=18, d_A=200, 5000 offline trajectories).
- A de Finetti theorem shows every exchangeable and coherent stateless decision process is a latent bandit.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ProBALL-UCB initialized with SOLD instead of LinUCB | Linear latent contextual bandits · Synthetic simulation | frequentist regret | No clear effect |
| Use tighter confidence bounds for Δ_off | ProBALL-UCB · Synthetic simulation | frequentist regret | Helped |
| Use ProBALL-UCB initialized with SOLD instead of LinUCB | MovieLens-1M · MovieLens-1M | frequentist regret | No clear effect |
| Use tighter confidence bounds for Δ_off | ProBALL-UCB · MovieLens-1M | frequentist regret | Helped |
| Use ProBALL-UCB instead of mUCB or mmUCB | MovieLens-1M recommendation · MovieLens-1M | frequentist regret | Mixed |
| Vary the hyperparameter τ across reasonable values (ablation) | ProBALL-UCB · Experiments across settings | frequentist regret | No clear effect |
| Use tighter rather than looser confidence-bound constructions (ablation) | ProBALL-UCB · Experiments across settings | frequentist regret | Helped |
| Increase the number of offline samples used by SOLD (ablation) | SOLD and ProBALL-UCB · MovieLens-1M | subspace recovery | Helped |
Surprising findings
- Coherence (equality in value, not just distribution) is necessary: exchangeability alone does not imply latent bandit structure.
- Naive PCA on single reward estimates or raw rewards gives erroneous subspaces because per-reward noise makes the target full rank.
Experimental setup
Datasets: MovieLens-1M
Benchmarks: MovieLens-1M simulated latent linear bandit
Metrics: frequentist cumulative regret
Baselines: LinUCB mUCB mmUCB TS mmTS MixTS
Builds on: LinUCB latent bandits probabilistic matrix factorization de Finetti theorem
Closest papers by meaning
- Regret minimization in Linear Bandits with offline data via extended D-optimal exploration. (NeurIPS 2026 · similarity 0.92)
- Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits (NeurIPS 2026 · similarity 0.91)
- Leveraging (Biased) Information: Multi-armed Bandits with Offline Data (ICML 2024 · similarity 0.89)
- Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach (AAAI 2024 · similarity 0.88)
- On The Statistical Complexity of Offline Decision-Making (ICML 2024 · similarity 0.88)
- Block Optimism for Nonstationary Bandits with Latent Linear Dynamics (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.