AI Paper Atlas

Research map / Bandits and online learning / RL theory and MDPs

Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPs

Max Simchowitz, Kevin G. Jamieson

NeurIPS 2019 · Bandits and online learning › RL theory and MDPs · approach: Regret bounds with function approximation · 24 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use StrongEuler with gap-dependent clipped regret analysisStrongEuler on episodic tabular MDPs · Episodic tabular MDPscumulative regret: C_{\mathcal{M}} \log TImproved
Interpolate the gap-dependent bound with the minimax regimeStrongEuler on episodic tabular MDPs · Episodic tabular MDPscumulative regret: \sqrt{HSAT \log(T) \log(\tfrac{MT}{\delta})} + H^4 S A (S \vee H) \log^2 \tfrac{TM}{\delta}Improved
Exploit state-action gaps and clip small optimistic surplusesOptimistic algorithms for episodic MDPs · Episodic tabular MDPscumulative regret: C_{\mathcal{M}} \log TImproved
Apply the StrongEuler bound to contextual bandit instancesStrongEuler · Contextual bandit instancescumulative regret: the factors of H under the square roots can be refined to a 1Improved
Apply the StrongEuler bound to MDPs with bounded rewardsStrongEuler · MDPs with \mathcal{G}-bounded rewards, \mathcal{G} \lesssim 1cumulative regret: the factors of H under the square roots can be replaced by a 1/HImproved
Rely on optimism in the stated worst-case instanceExisting optimistic algorithms for finite-horizon MDPs · MDPs with one gap \mathtt{gap}_{\mathtt{min}}=\epsilon and other state gaps at least 1/2regret lower bound: \frac{S}{\mathtt{gap}_{\mathtt{min}}}\log (1/\delta)Hurt
Use optimism without a diameter or ergodicity dependence in the boundStrongEuler · Episodic tabular MDPsregret bound dependence: depend only on H and not on the diameter DImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy