Research map / Bandits and online learning / Contextual bandits and Thompson sampling
RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RoME, a robust mixed-effects Thompson-sampling contextual bandit for mHealth that combines user/time random effects, network cohesion penalties, and debiased machine learning for the baseline reward, with a high-probability regret bound depending only on the differential-reward dimension.
Problem
Mobile health bandits must learn treatment effects that vary across users and time under nonstationary, nonlinear baseline rewards, with participant heterogeneity and limited data per user.
Approach
Model the differential reward with shared fixed effects plus user- and time-specific random effects, regularize them with nearest-neighbor (network cohesion) Laplacian penalties, and build doubly robust pseudo-rewards using DML-fitted baseline models so regret depends only on the differential-reward dimension.
Key results
- In the Nonlinear simulation setting, RoME outperformed each competitor in at least 48 of 50 repetitions and had the lowest cumulative regret by stage 200.
- On the Valentine Study off-policy evaluation, RoME achieved significantly higher average reward than the other five algorithms (p <= 0.01), implying about a 3.5% step-count increase over constant randomization.
- The regret bound is of order d*sqrt(K log K log(K^2 d)){sqrt(d)+log^{1/4}(K)}, depending only on the differential-reward dimension d, scaling as roughly sqrt(K) d^{3/2} up to logs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RoME instead of the competing methods | mHealth contextual bandits · Heterogeneous Users simulation | cumulative regret | Helped |
| Use RoME instead of the competing methods | mHealth contextual bandits · Nonlinear simulation | cumulative regret | Helped |
| Use RoME instead of IntelPooling | mHealth contextual bandits · Homogeneous Users simulation | cumulative regret | No clear effect |
| Use RoME instead of the constant randomization policy | Valentine Study intervention policy · Valentine Study | step count: +3.5% | Helped |
| Replace the bagged stochastic gradient tree baseline model with bagged linear models (ablation) | RoME · Nonlinear simulation | cumulative regret | No clear effect |
| Add user-specific advantage functions (ablation) | RoME-SU · Heterogeneous Users simulation | cumulative regret | Helped |
| Add user-specific advantage functions (ablation) | RoME-SU · Nonlinear simulation | final regret win rate versus RoME: 100% | Helped |
| Add DML and time effects (ablation) | NNR-Linear · Nonlinear simulation | final regret win rate versus RoME: 96% | Helped |
| Increase the number of neighbors to 10 (ablation) | RoME · Heterogeneous Users simulation | final regret win rate versus RoME-SU: 80% to 88% | Helped |
| Rescale gamma by a factor of ten (ablation) | RoME · Heterogeneous Users simulation | final regret win rate versus NNR-Linear: 56% to 70% | Helped |
| Rescale lambda by a factor of ten (ablation) | RoME · Heterogeneous Users simulation | relative performance | Helped |
| Add low or medium noise to the nearest-neighbor network (ablation) | RoME · Nonlinear simulation | cumulative regret | Helped |
Surprising findings
- RoME-BLM, using misspecified bagged linear models for the baseline, performs statistically indistinguishably from RoME with nonlinear bagged trees, suggesting robustness to baseline misspecification.
- In the Homogeneous Users setting, RoME does not outperform IntelPooling and is competitive but not superior.
Limitations
- Theoretical assumptions (e.g., on V0 theta term and minimum eigenvalue growth) may not be plausible for more than two arms (q>1).
- Computation may need approximations for large-scale commercial deployment.
- Long-term effects such as treatment fatigue are not modeled.
Experimental setup
Datasets: simulated mHealth data Valentine Study Intern Health Study
Benchmarks: simulated mHealth bandit (Homogeneous, Heterogeneous, Nonlinear settings) Valentine Study off-policy evaluation Intern Health Study off-policy evaluation
Metrics: cumulative regret average per-trial reward improvement pairwise win percentage paired t-test p-values computation time
Baselines: Standard Thompson sampling AC (action-centered contextual bandit) IntelPooling Neural-Linear NNR-Linear RoME-SU RoME-BLM
Builds on: action-centered contextual bandit nearest-neighbor regularization intelligentpooling debiased machine learning linear Thompson sampling
Closest papers by meaning
- Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions (NeurIPS 2025 · similarity 0.86)
- A Classification View on Meta Learning Bandits (ICML 2025 · similarity 0.86)
- Mixed-Effects Contextual Bandits (AAAI 2024 · similarity 0.86)
- Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits (AAAI 2025 · similarity 0.86)
- Mutli-Armed Bandits with Network Interference (NeurIPS 2024 · similarity 0.84)
- A General Framework for Off-Policy Learning with Partially-Observed Reward (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.