Research map / Bandits and online learning / Contextual bandits and Thompson sampling
Impatient Bandits: Optimizing for the Long-Term Without Delay
Kelly W. Zhang, Thomas Baldwin-McDonald, Kamil Ciosek, Lucas Maystre, Daniel Russo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use progressive (intermediate) feedback via Bayesian filter instead of waiting for delayed reward (ablation) | Impatient Bandit (Thompson sampling) · Synthetic environment (20 actions, J=25) | cumulative regret | Improved |
| reduce batch size m from 1000 to 10 (ablation) | Impatient Bandit · Synthetic environment (20 actions, J=25) | cumulative regret | No clear effect |
| increase informativeness of intermediate outcomes (alpha 0.1 to 0.4 to 0.8) (ablation) | Impatient Bandit vs delayed Thompson sampling · Synthetic environment | log regret ratio vs delayed TS | Improved |
| increase batch size m used to fit prior on Spotify validation data (ablation) | Fitted Bayesian predictive model · Spotify podcast validation set | mean absolute error of reward prediction | Improved |
| observe more days of engagement before predicting 60-day reward (ablation) | Fitted Bayesian predictive model · Spotify podcast validation set | mean absolute error of reward prediction | Improved |
| increase batch size m (ablation) | Value of Progressive Feedback (fitted prior) · Spotify podcast data | estimated VoPF | Improved |
Closest papers by meaning
- Fighting Boredom in Recommender Systems with Linear Reinforcement Learning (NeurIPS 2018 · similarity 0.90)
- Non-Stationary Delayed Bandits with Intermediate Observations (ICML 2020 · similarity 0.90)
- Cascading Bandits: Optimizing Recommendation Frequency in Delayed Feedback Environments (NeurIPS 2023 · similarity 0.90)
- Bandit Learning with Joint Effect of Incentivized Sampling, Delayed Sampling Feedback, and Self-Reinforcing User Preferences (ICLR 2022 · similarity 0.90)
- Bandits Meet Mechanism Design to Combat Clickbait in Online Recommendation (ICLR 2024 · similarity 0.89)
- On the Unexpected Effectiveness of Reinforcement Learning for Sequential Recommendation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
