Research map / Reinforcement learning / Policy gradient and actor-critic
Harnessing Structures for Value-Based Planning and Reinforcement Learning
Yuzhe Yang, Guo Zhang, Zhi Xu, Dina Katabi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVP with 20% observation probability instead of the optimal policy | Inverted Pendulum, 400×100 discretization · Inverted Pendulum | average deviation (degree): 1.49 -> 2.07 | Hurt |
| Use SVP with 20% observation probability instead of the optimal policy | Inverted Pendulum, 2500×1000 discretization · Inverted Pendulum | average deviation (degree): 0.53 -> 1.92 | Hurt |
| Use SVP with 20% observation probability instead of the optimal policy | Inverted Pendulum, 10000×4000 discretization · Inverted Pendulum | average deviation (degree): 0.18 -> 0.96 | Hurt |
| Apply SV-RL to low-rank Atari games rather than the vanilla value-based algorithm | DQN, double DQN, and dueling DQN · Low-rank Atari games | mean reward | Improved |
| Apply SV-RL to higher-rank games | Value-based Atari methods · High-rank Atari games, including Seaquest | mean reward | Mixed |
| Increase SV-RL batch size from 32 to 128 (ablation) | DQN · Seaquest | performance | Improved |
| Use a smaller batch size (ablation) | SV-RL · Seaquest | performance | Hurt |
Closest papers by meaning
- Stochastic Q-learning for Large Discrete Action Spaces (ICML 2024 · similarity 0.90)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.89)
- On Value Function Representation of Long Horizon Problems (AAAI 2018 · similarity 0.89)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.89)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.88)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
