Research map / Bandits and online learning / Online convex optimization and games
Non-Stationary Delayed Bandits with Intermediate Observations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use intermediate observations to estimate changing transitions and delayed rewards separately | NSD-UCRL2 · non-stationary delayed bandits; T=8000, Γ_T=3, D∈{100, 500, 1000} | cumulative regret | Helped |
| use intermediate observations rather than ignoring them | NSD-UCRL2 versus UCB and SW-UCB · non-stationary delayed bandits; D∈{100, 500, 1000} | cumulative regret | Helped |
| use intermediate observations rather than ignoring them | NSD-UCRL2 versus Oracle-UCB · non-stationary delayed bandits with large delays | cumulative regret | Helped |
| set sliding window too small (W=400) (ablation) | NSD-UCRL2 · non-delayed non-stationary bandits; T=8000, Γ_T=3 | cumulative regret | Hurt |
| increase the sliding window from W=400 toward W=800 (ablation) | NSD-UCRL2 · non-delayed non-stationary bandits; T=8000, Γ_T=3 | cumulative regret | Helped |
| set the sliding window above 1000 (ablation) | NSD-UCRL2 · non-delayed non-stationary bandits; T=8000, Γ_T=3 | cumulative regret | Mixed |
| introduce low-level reward-model mixing (α=0.1), preserving the best arm (ablation) | NSD-UCRL2 · stationary bandit with actions 1 and 2 | cumulative regret | No clear effect |
| introduce low-level reward-model mixing (α=0.1), with the best arm not preserved (ablation) | NSD-UCRL2 · stationary bandit with actions 1 and 2 | cumulative regret | No clear effect |
| increase mixing to α=0.3 while preserving the best arm (ablation) | NSD-UCRL2 · stationary bandit with actions 1 and 2 | cumulative regret | Mixed |
| increase mixing to α=0.3 so the best arm changes (ablation) | NSD-UCRL2 · stationary bandit with actions 1 and 2 | cumulative regret | Hurt |
| increase mixing in the non-stationary delayed setting (ablation) | NSD-UCRL2 · misspecified non-stationary delayed bandits; D=500 | cumulative regret | Mixed |
Closest papers by meaning
- Non-stationary Bandit Convex Optimization: A Comprehensive Study (NeurIPS 2025 · similarity 0.89)
- Near-Optimal Stochastic Linear Bandits with Delay (NeurIPS 2026 · similarity 0.89)
- A Best-of-both-worlds Algorithm for Bandits with Delayed Feedback with Robustness to Excessive Delays (NeurIPS 2024 · similarity 0.88)
- Online Nonsubmodular Optimization with Delayed Feedback in the Bandit Setting (AAAI 2025 · similarity 0.87)
- Constrained Feedback Learning for Non-Stationary Multi-Armed Bandits (NeurIPS 2025 · similarity 0.87)
- Bandit and Delayed Feedback in Online Structured Prediction (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.