Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a two-year software system that runs contextual Thompson Sampling bandits alongside uniform random assignment in a text-message mental health intervention, evaluated via simulations and a 1100-user deployment.
Problem
Adaptive bandit experiments can improve user reward in digital mental health messaging, but it is unclear when they also support statistically valid analysis by domain scientists (bias, false positives, power loss).
Approach
A system that instruments text-message intervention components as contextual bandit decision points, randomly alternating each assignment between contextual Thompson Sampling and uniform random to compare both side by side, with simulations of FPR and power in addition to real deployment rewards.
Key results
- In simulation, Contextual TS gets about 0.14 more average reward than Uniform Random for participants in good mood when arms differ.
- Contextual TS FPR was 0.07 vs 0.04 for Uniform Random at N=1000 in the no-difference scenario; in Scenario 2 power at N=100 was 0.72 vs 0.87 for Uniform Random.
- Real deployment yielded 813 ratings from 8,521 arm assignments (9.54% engagement), with Contextual TS showing higher mean reward than Uniform Random for both Link and Rationale levels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Simulation, Scenario 1, N=1000 | false positive rate: 0.04 -> 0.07 | Hurt |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Simulation, Scenario 2, N=100 | statistical power for Rationale: 0.87 -> 0.72 | Hurt |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Simulation, Scenario 3 | statistical power: Rationale: 1.00 -> 0.92; Mood: 0.95 -> 0.41; Rationale * Mood: 1.00 -> 1.00 (N=100); Rationale: 1.00 -> 1.00; Mood: 1.00 | Mixed |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Real-world deployment, Link present | mean reward: 0.719 (0.021) -> 0.790 (0.018) | Helped |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Real-world deployment, Link absent | mean reward: 0.640 (0.024) -> 0.716 (0.024) | Helped |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Real-world deployment, Rationale present | mean reward: 0.710 (0.022) -> 0.736 (0.018) | Helped |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Real-world deployment, Rationale absent | mean reward: 0.703 (0.024) -> 0.728 (0.025) | Helped |
| Use Contextual TS instead of Uniform Random | Adaptive message assignment · Real-world deployment, Link response rate | response rate: Link present: 22%; Link absent: 20% | No clear effect |
Surprising findings
- Contextual TS had lower power for detecting the single Mood effect in Scenario 3 because it concentrates assignments, while uniform random estimates it more accurately.
Limitations
- Real-world deployment engagement is low (about 9.5% of assignments received ratings), limiting reward signal.
- Mood and activity contextual effects are checked only descriptively due to low power in early stages.
- Simulation settings are simplified (Gaussian rewards discretized to five levels, single contextual variable in Scenario 3).
Experimental setup
Datasets: real-world deployment data from 1100 users recruited via Mental Health America simulated data under three scenarios
Benchmarks: simulation scenarios 1-3 (FPR, power, reward) 1100-user real-world deployment (engagement, mean reward)
Metrics: mean reward false positive rate statistical power engagement rate
Baselines: Uniform Random assignment
Builds on: Thompson Sampling contextual multi-armed bandit MOOClet
Closest papers by meaning
- Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits (AAAI 2025 · similarity 0.84)
- RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions (NeurIPS 2024 · similarity 0.84)
- Adaptive Experimentation When You Can't Experiment (NeurIPS 2024 · similarity 0.82)
- Benchmarking Estimators for Natural Experiments: A Novel Dataset and a Doubly Robust Algorithm (NeurIPS 2024 · similarity 0.82)
- MindGuard: Guardrail Classifiers for Multi-Turn Mental Health Support (NeurIPS 2026 · similarity 0.82)
- Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.