Research map / Reinforcement learning / Policy gradient and actor-critic
Q-functionals for Value-Based Continuous Control
Samuel Lobel, Sreehari Rammohan, Bowen He, Shangqun Yu, George Konidaris
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace policy-network action selection with choosing the best of 1,000 random actions | Q-functional on LunarLanderContinuous-v2 · LunarLanderContinuous-v2 | fraction of states where selected action beats one random action: 99.9% | Improved |
| Use the trained DDPG policy for action selection rather than random sampling | DDPG on LunarLanderContinuous-v2 · LunarLanderContinuous-v2 | fraction of states where selected action is below a random action: 15% | Hurt |
| Replace repeated state-action network evaluations with a rank 3 Legendre Q-functional | Q-value evaluation on Hopper · Hopper | action-evaluation speed: 3.5 times | Improved |
| Precompute action basis representations (ablation) | Q-functionals · Hopper | action-evaluation speed: more than 20 times | Improved |
| Precompute action basis representations (ablation) | Q-functionals · continuous-control tasks | reinforcement-learning behavior | No clear effect |
| Increase the number of samples used to select the maximum-value action (ablation) | Q-functional on Humanoid-v2 · Humanoid-v2 | performance: 1,000 | Improved |
| Represent a policy with a Q-value-derived distribution instead of a squashed Gaussian | Q-functional versus SAC on the bandit task · bandit task | ability to represent a bimodal optimal policy | Improved |
Closest papers by meaning
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.92)
- Stochastic Q-learning for Large Discrete Action Spaces (ICML 2024 · similarity 0.90)
- Learning from Suboptimal Data in Continuous Control via Auto-Regressive Soft Q-Network (ICML 2025 · similarity 0.90)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.89)
- Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Continuous Q-Score Matching: Diffusion Guided Reinforcement Learning for Continuous-Time Control (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
