Research map / Reinforcement learning / Policy gradient and actor-critic
Value Function Decomposition for Iterative Design of Reinforcement Learning Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add value decomposition to SAC without twin-network correction or CAGrad (SAC-D-Naive) (ablation) | SAC · Continuous-action Gym benchmark environments | episode return | Hurt |
| Use the minimum composite Q-network and take all component predictions from that network, rather than applying an element-wise minimum (ablation) | SAC-D · Continuous-action Gym benchmark environments | episode return | Helped |
| Add CAGrad to value-decomposed SAC (ablation) | SAC-D · Continuous-action Gym benchmark environments | episode return | Helped |
| Add CAGrad to value-decomposed SAC (ablation) | SAC-D · Ant-v3 | episode return | Helped |
| Add CAGrad to value-decomposed SAC (ablation) | SAC-D · BipedalWalkerHardcore-v3 | episode return | Helped |
| Add CAGrad to SAC-D (ablation) | SAC-D · BipedalWalkerHardcore-v3 | training steps/s: 142 (8) -> 26 (2) | Hurt |
| Add a zero-velocity trace observation feature (ablation) | LunarLander-v2 landing-value prediction · LunarLander-v2, landing prediction | RMSE: 5.61 (0.12) -> 1.05 (0.28) | Helped |
| Add a zero-velocity trace observation feature (ablation) | LunarLander-v2 landing-value prediction · LunarLander-v2, landing prediction | correlation: 0.521 (0.030) -> 0.996 (0.001) | Helped |
| Apply sign constraints to the crash component's target and value prediction (ablation) | SAC-D · LunarLander-v2 | policy learning performance | No clear effect |
| Schedule the failure reward weight from 0.01 to 1 during training (ablation) | SAC-D-CAGrad · BipedalWalkerHardcore-v3 | forward progress | Helped |
| Remove environment termination conditions (ablation) | SAC-D · Ant-v3 | episode return | Helped |
| Remove environment termination conditions (ablation) | SAC-D · Ant-v3 | critic loss | Helped |
Closest papers by meaning
- The Value-Improvement Path: Towards Better Representations for Reinforcement Learning (AAAI 2021 · similarity 0.90)
- Deconfounded Value Decomposition for Multi-Agent Reinforcement Learning (ICML 2022 · similarity 0.90)
- Value-Decomposition Multi-Agent Actor-Critics (AAAI 2021 · similarity 0.89)
- Parameter-Based Value Functions (ICLR 2021 · similarity 0.89)
- O-MAPL: Offline Multi-agent Preference Learning (ICML 2025 · similarity 0.88)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.