AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

General Munchausen Reinforcement Learning with Tsallis Kullback-Leibler Divergence

Lingwei Zhu, Zheng Chen, Matthew Kyle Schlegel, Martha White

NeurIPS 2023 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use Tsallis KL regularization with q=2 (MVI(q=2)) instead of standard KL (q=1)Munchausen Value Iteration on QR-DQN · Atari 2600 (about 35 games, 50M frames)game scoreImproved
Use MVI(q=2) instead of MVI(q=1)Munchausen Value Iteration · PrivateEye, Chopper and Seaquestgame scoreHurt
Add Munchausen (KL) term, alpha greater than 0, to Tsallis-VI (ablation)Tsallis-VI (entropy only) · 35 Atari gamesgame scoreImproved
Add Munchausen (KL) term to Tsallis-VI (ablation)Tsallis-VI · 10 Atari gamesgame score: more than 400% improvementImproved
Use q-logarithm of policy with action gap Q_k - M_q Q_k as Munchausen term instead of ln_q pi (ablation)MVI(q) · CartPole-v1learning stabilityImproved
Increase q from 2 to 5 in the Munchausen approximation (ablation)MVI(q) · CartPole-v1residual term magnitudeHurt
Simply replace ln pi with ln_q pi in MVI (ablation)MVI · CartPole-v1learning curveHurt
Use Tsallis entropy alone (q=2 sparsemax) without KLpolicy optimization · prior work comparisonperformanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy