Research map / Reinforcement learning / Policy gradient and actor-critic
Marginal Utility for Planning in Continuous or Large Discrete Action Spaces
Zaheen Ahmad, Levi Lelis, Michael Bowling
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the candidate action generator with marginal utility instead of a learned policy objective | Curling action generation with UCB · Curling hammer states | expected points: over 0.15 points per hammer state | Improved |
| Initialize search with candidates from the marginal utility generator rather than generators trained with max, softmax, or sum objectives | Curling action generation with UCB and KR-UCB · Curling hammer states | expected points | Improved |
| Train the candidate generator with max utility instead of marginal utility (ablation) | Curling action generation · Curling hammer states | expected points | Hurt |
| Train the candidate generator with softmax utility instead of marginal utility (ablation) | Curling action generation · Curling hammer states | expected points | Hurt |
| Train the candidate generator with sum utility instead of marginal utility (ablation) | Curling action generation · Curling hammer states | expected points | Hurt |
| Use marginal utility rather than max utility to train the candidate generator (ablation) | Curling action generation · Curling hammer states | expected points | Improved |
| Initialize KR-UCB with marginal-utility candidates rather than candidates from other objectives | KR-UCB · Curling hammer states at smaller search budgets | expected points | Improved |
| Train the candidate generator with marginal utility rather than softmax utility (ablation) | Location game action generation · Location game | expected utility | Improved |
| Train the candidate generator with marginal utility rather than sum utility using REINFORCE (ablation) | Location game action generation · Location game | expected utility | Improved |
| Train the candidate generator with max utility rather than marginal utility (ablation) | Location game action generation · Location game | expected utility | No clear effect |
Closest papers by meaning
- ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning (AAAI 2024 · similarity 0.87)
- Learning Planning Domains from Non-redundant Fully-Observed Traces: Theoretical Foundations and Complexity Analysis (AAAI 2024 · similarity 0.87)
- Reward-Respecting Subtasks for Model-Based Reinforcement Learning (Abstract Reprint) (AAAI 2024 · similarity 0.86)
- Reflective Prompted Policy Optimization: Trajectory-Grounded Revision and Salience Bias (NeurIPS 2026 · similarity 0.86)
- Marginal Benefit Driven RL Teacher for Unsupervised Environment Design (AAAI 2025 · similarity 0.86)
- A Minimaximalist Approach to Reinforcement Learning from Human Feedback (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
