AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

An Intrinsically-Motivated Approach for Learning Highly Exploring and Fast Mixing Policies

Mirco Mutti, Marcello Restelli

AAAI 2020 · Reinforcement learning › MDPs and planning · approach: Rich-observation exploration · 15 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use the Frobenius objective rather than the Infinity or Column Sum objectives in IDE³AL (ablation)IDE³AL · Single Chain, Double Chain, and Knight Queststate distribution entropy: Single Chain: 0.983 / 0.985 / 0.984; Double Chain: 0.967 / 0.961 / 0.970; Knight Quest: 0.720 / 0.837 / 0.850Mixed
use the Frobenius formulation rather than the alternative objective formulations during learning (ablation)IDE³AL · Single Chainstate distribution entropyImproved
iteratively alternate model estimation with optimization sweeps instead of collecting random-policy samples before offline optimization (ablation)IDE³AL · Double Chaintransition model estimation errorImproved
use IDE³AL instead of CountBasedexploration policies · Double Chainstate entropyImproved
use IDE³AL instead of MaxEntexploration policies · Double Chain and Single Chainconvergence speedImproved
use CountBased instead of IDE³ALexploration policies · Single Chainexploration performanceNo clear effect
use IDE³AL instead of MaxEntexploration policies · Knight Queststate entropyImproved
add a minimum action-probability constraint (ablation)state-entropy exploration objective · Single Chainstate distribution entropyHurt
constrain the target transition matrix with ζ to prioritize faster mixing (ablation)IDE³AL Infinity and Frobenius objectives · Single Chainstate distribution entropyMixed
use the uniform transition matrix as the target (ablation)IDE³AL mixing objective · Single Chainspectral gapImproved
maximize entropy directly over the stationary state-action distribution rather than using a lower-bound formulation (ablation)Dual formulation · Single Chain, Double Chain, and Knight Queststate distribution entropy: Single Chain: 0.985 vs 0.984; Double Chain: 0.971 vs 0.970; Knight Quest: 0.859 vs 0.850No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy