AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

QUOTA: The Quantile Option Architecture for Reinforcement Learning

Shangtong Zhang, Hengshuai Yao

AAAI 2019 · Reinforcement learning › MDPs and planning · approach: Average-reward and regularized MDPs · 24 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Select actions using learned quantile options rather than only the meanQR-DQN · Atari, 49 gamesfinal performance: outperformed QR-DQN in 23 games and underperformed QR-DQN in 14 gamesMixed
Select actions using learned quantile options rather than only the meanQR-DQN · Atari, 49 gamescumulative rewards during training: outperformed QR-DQN in 21 games and underperformed QR-DQN in 13 gamesMixed
Select actions using learned quantile options rather than only the meanQR-DQN · 10 most challenging Atari gamescumulative rewards: 97.2% cumulative reward improvement in averageImproved
Use learned quantile options instead of a matched increase in ε explorationQR-DQN-Alt · Atari, 49 gamesfinal performance: outperformed QR-DQN-Alt in 27 games and underperformed QR-DQN-Alt in 14 gamesMixed
Use learned quantile options instead of a matched increase in ε explorationQR-DQN-Alt · Atari, 49 gamescumulative rewards during training: outperformed QR-DQN-Alt in 42 games and underperformed QR-DQN-Alt in 5 gamesMixed
Add adaptive selection among quantile-based actorsQR-DDPG and DDPG · Roboschoolperformance: 5 out of the 6 tasksImproved
Use an optimistic high quantile for behaviorQR in Chain 1 · Chain 1steps to find optimal policyImproved
Use a pessimistic low quantile for behaviorQR in Chain 2 · Chain 2steps to find optimal policyImproved
Use a low quantile rather than a high quantile for behaviorQR in Chain 1 · Chain 1steps to find optimal policyHurt
Use a high quantile rather than a low quantile for behaviorQR in Chain 2 · Chain 2steps to find optimal policyHurt
Increase ε exploration using a linear decay to zero (ablation)QR-DQN-Alt · Atari, compared with QUOTAfinal performance: 27 / 42 games; underperformed in 14 / 5 gamesMixed

Later papers that compared with this method

1 later accepted paper reports QUOTA as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy