Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives non-parametric causal bounds on conditional causal effects under confounding and selection bias from biased offline data, and uses them to clip UCB indices in LinUCB-PCB and UCB-PCB to reduce regret.
Problem
Offline observational data for bandits carry compound confounding and sample selection biases, so naively fitting reward estimates from them can hurt online bandit learning when causal effects are unidentifiable or nonrecoverable.
Approach
Bound the conditional interventional reward for each arm using c-component factorization (with RC* recovery) and substitute interventions under selection bias, take the tighter bounds, and truncate UCB indices with these bounds so suboptimal arms are excluded, with regret guarantees.
Key results
- BCE causal bounds contain the ground-truth causal effect for each of 16 context values, while Biased and CP estimates deviate under compound biases.
- LinUCB-PCB achieves the lowest cumulative regret among compared methods in the synthetic linear contextual bandit experiment (averaged over 100 simulations).
- LinUCB-Biased and LinUCB-CP perform worse than the plain LinUCB baseline, showing naive use of biased offline estimates can hurt online learning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use prior causal bounds from compound-bias offline data to guide arm selection | LinUCB · Synthetic contextual linear bandit | cumulative regret | Helped |
| Initialize with offline estimates accounting only for confounding bias | LinUCB · Synthetic contextual linear bandit | cumulative regret | Hurt |
| Initialize with naive conditional probability estimates that ignore both biases | LinUCB · Synthetic contextual linear bandit | cumulative regret | Hurt |
Surprising findings
- Naively initializing LinUCB with biased offline estimates (Biased, CP) performs worse than plain LinUCB with no offline data.
Limitations
- When c-components yield no recoverable factors, bounds fall back to the loose [0,1] interval under semi-Markovian models.
- Empirical evaluation limited to a single synthetic linear contextual bandit setting.
- Assumes the causal graph is known and invariant across offline and online phases.
Experimental setup
Datasets: synthetic data generated from a causal graph with confounding and selection bias
Benchmarks: synthetic confounded and selection-biased bandit data
Metrics: causal bound coverage of ground-truth causal effect cumulative regret
Baselines: LinUCB LinUCB-Biased LinUCB-CP UCB
Builds on: LinUCB UCB RC algorithm generalized adjustment for selection bias cond-identify
Closest papers by meaning
- Causal Bandits: The Pareto Optimal Frontier of Adaptivity, a Reduction to Linear Bandits, and Limitations around Unknown Marginals (ICML 2024 · similarity 0.92)
- Leveraging (Biased) Information: Multi-armed Bandits with Offline Data (ICML 2024 · similarity 0.90)
- Partial Structure Discovery is Sufficient for No-regret Learning in Causal Bandits (NeurIPS 2024 · similarity 0.90)
- Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits (NeurIPS 2026 · similarity 0.89)
- Regret minimization in Linear Bandits with offline data via extended D-optimal exploration. (NeurIPS 2026 · similarity 0.89)
- Leveraging Offline Data in Linear Latent Contextual Bandits (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.