Research map / Reinforcement learning / MDPs and planning
QUOTA: The Quantile Option Architecture for Reinforcement Learning
Shangtong Zhang, Hengshuai Yao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select actions using learned quantile options rather than only the mean | QR-DQN · Atari, 49 games | final performance: outperformed QR-DQN in 23 games and underperformed QR-DQN in 14 games | Mixed |
| Select actions using learned quantile options rather than only the mean | QR-DQN · Atari, 49 games | cumulative rewards during training: outperformed QR-DQN in 21 games and underperformed QR-DQN in 13 games | Mixed |
| Select actions using learned quantile options rather than only the mean | QR-DQN · 10 most challenging Atari games | cumulative rewards: 97.2% cumulative reward improvement in average | Improved |
| Use learned quantile options instead of a matched increase in ε exploration | QR-DQN-Alt · Atari, 49 games | final performance: outperformed QR-DQN-Alt in 27 games and underperformed QR-DQN-Alt in 14 games | Mixed |
| Use learned quantile options instead of a matched increase in ε exploration | QR-DQN-Alt · Atari, 49 games | cumulative rewards during training: outperformed QR-DQN-Alt in 42 games and underperformed QR-DQN-Alt in 5 games | Mixed |
| Add adaptive selection among quantile-based actors | QR-DDPG and DDPG · Roboschool | performance: 5 out of the 6 tasks | Improved |
| Use an optimistic high quantile for behavior | QR in Chain 1 · Chain 1 | steps to find optimal policy | Improved |
| Use a pessimistic low quantile for behavior | QR in Chain 2 · Chain 2 | steps to find optimal policy | Improved |
| Use a low quantile rather than a high quantile for behavior | QR in Chain 1 · Chain 1 | steps to find optimal policy | Hurt |
| Use a high quantile rather than a low quantile for behavior | QR in Chain 2 · Chain 2 | steps to find optimal policy | Hurt |
| Increase ε exploration using a linear decay to zero (ablation) | QR-DQN-Alt · Atari, compared with QUOTA | final performance: 27 / 42 games; underperformed in 14 / 5 games | Mixed |
Later papers that compared with this method
1 later accepted paper reports QUOTA as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion (NeurIPS 2023 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learning (AAAI 2025 · similarity 0.87)
- Quantile Geometry Regularization for Distributional Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- Minimax Optimal Reinforcement Learning with Quasi-Optimism (ICLR 2025 · similarity 0.86)
- Beyond Optimism: Exploration With Partially Observable Rewards (NeurIPS 2024 · similarity 0.86)
- Risk-Conditioned Reinforcement Learning: A Generalized Approach for Adapting to Varying Risk Measures (AAAI 2024 · similarity 0.86)
- Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
