Research map / Bandits and online learning / Multi-armed bandits
A Closer Look at the Worst-case Behavior of Multi-armed Bandit Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use canonical UCB with ρ > 1 | Two-armed MAB with small gap · Small gap, Δ = o(√(log n/n)) | optimal-arm sampling rate: N_{i*}(n)/n → 1/2 in probability | Helped |
| Use canonical UCB with ρ > 1 | Two-armed MAB · Large gap, Δ = ω(√(log n/n)) | optimal-arm sampling rate: N_{i*}(n)/n → 1 in probability | Helped |
| Scale the gap as Δ ∼ √(θ log n/n) | UCB (ρ > 1) · Two-armed MAB, moderate gap | optimal-arm sampling rate: N_{i*}(n)/n → λ*ρ(θ) in probability; λ*ρ(0) = 1/2; λ*ρ(θ) → 1 as θ → ∞ | Mixed |
| Use UCB when every arm is optimal | K-armed MAB · All arms optimal | per-arm sampling rate: N_i(n)/n → 1/K in probability | Helped |
| Use UCB when optimal arms are well-separated from inferior arms | K-armed MAB · Well-separated optimal arms | per-optimal-arm sampling rate: N_i(n)/n → 1/|I| in probability | Helped |
| Use Beta-prior Thompson Sampling rather than UCB for identical Bernoulli arms with q = 1 | Two-armed MAB · Bernoulli(q) arms, q = 1 | arm-sampling balance: N_1(n)/n ⇒ Uniform distribution on [0,1] | Hurt |
| Use Beta-prior Thompson Sampling on identical Bernoulli arms with q = 0 | Two-armed MAB · Bernoulli(q) arms, q = 0 | arm 1 sampling rate: N_1(n)/n ⇒ 1/2 | No clear effect |
| Use canonical UCB with ρ > 1 in the moderate-gap regime | Two-armed MAB · Δ ∼ √(θ log n/n) | stochastic regret: R_n^π/√(n log n) ⇒ h_ρ(θ) | Hurt |
| Use canonical UCB with ρ > 1 at diffusion-scale gaps | Two-armed MAB · Δ = c/√n | normalized regret process: R^π_{⌊nt⌋}/√n ⇒ Δ₀t/2 + √((σ₁²+σ₂²)/2) B̃(t) | Mixed |
| Increase the UCB exploration coefficient ρ | UCB · Moderate-gap worst-case regret | maximum asymptotic regret constant: 0.2367, 0.3192, 0.3909, 0.4514 | Hurt |
Closest papers by meaning
- Lenient Regret for Multi-Armed Bandits (AAAI 2021 · similarity 0.92)
- Optimal Estimation of the Best Mean in Multi-Armed Bandits (NeurIPS 2025 · similarity 0.92)
- p-Mean Regret for Stochastic Bandits (AAAI 2025 · similarity 0.91)
- Forced Exploration in Bandit Problems (AAAI 2024 · similarity 0.91)
- Bayesian Regret Minimization in Offline Bandits (ICML 2024 · similarity 0.91)
- Nash Social Welfare for Multi Armed Bandits: Trajectory-wise Expected and High Probability Regret (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.