Research map / Bandits and online learning / Contextual bandits and Thompson sampling

Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits

NeurIPS 2026 · Bandits and online learning › Contextual bandits and Thompson sampling · approach: Practical contextual bandit extensions · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a directional bias certificate (M_bias, rho) for offline-to-online linear contextual bandits and the Ellipsoidal-MINUCB algorithm that pools offline and online data with an online fallback, plus an adaptive epoch-wise estimate of the certificate.

Problem

Using biased offline logs to reduce online exploration in linear contextual bandits when the offline and online parameters differ; prior scalar Euclidean bias bounds are too coarse to capture which directions matter.

Approach

Replace a Euclidean bias radius with a matrix-weighted directional certificate ||theta_* - theta_off||_{M_bias} <= rho, use it to build an offline-pooled UCB/LCB branch, and take the min with the standard online UCB so misleading offline data cannot hurt beyond the online fallback; estimate the certificate adaptively from offline and online data per epoch.

Key results

• Theorem gives regret min{C_SL sqrt(T d L^3), pooled bound}, so the method matches SupLinUCB worst-case rate up to logs.
• Lower bound: diagonal Gaussian subclass shows transfer hardness governed by subset offline coverage N(S) and directional budgets v(S).
• Impossibility: a Euclidean radius bound alone forces Omega(rho^2 T/rho_0) regret for an oracle knowing the offline parameter, while a directional instance is easy.

Surprising findings

• Even with the offline parameter known exactly, a single scalar Euclidean bias bound can force dimension-dependent regret.

Limitations

• Certificate (M_bias, rho) must be valid; if undersized the pooled branch is only a heuristic.
• Experiments are small simulations; a 3D variant and adaptive variant show extra conservatism.
• Lower bound is established only for a diagonal Gaussian subclass; non-diagonal case is open.

Experimental setup

Benchmarks: simulated 5-arm 5-dimensional linear bandit

Metrics: cumulative regret

Baselines: SupLinUCB warm-started SupLinUCB offline-only policy MIN-UCB

Builds on: SupLinUCB OFUL MIN-UCB

Closest papers by meaning

• Leveraging (Biased) Information: Multi-armed Bandits with Offline Data (ICML 2024 · similarity 0.91)
• Leveraging Offline Data in Linear Latent Contextual Bandits (ICML 2025 · similarity 0.91)
• Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach (AAAI 2024 · similarity 0.89)
• Regret minimization in Linear Bandits with offline data via extended D-optimal exploration. (NeurIPS 2026 · similarity 0.89)
• Offline-to-Online Hyperparameter Transfer for Stochastic Bandits (AAAI 2025 · similarity 0.88)
• Information-Directed Offline-to-Online Reinforcement Learning (NeurIPS 2026 · similarity 0.88)