Research map / Reinforcement learning / Policy gradient and actor-critic
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the critic with the decision-aware loss instead of squared Q loss (MSE) | decision-aware actor-critic, linear actor and critic · Cliff World and Frozen Lake, direct and softmax representations | policy return J(pi) | Helped |
| train the critic with the decision-aware loss instead of advantage squared loss (Adv-MSE) | decision-aware actor-critic, linear actor and critic · Cliff World, d = 40 | policy return J(pi) | Helped |
| use Adv-MSE critic loss instead of MSE | actor-critic with linear critic · Cliff World and Frozen Lake, d = 40 and 60 | policy return J(pi) | Helped |
| increase critic feature dimension d from 40 to 80 (more expressive critic) (ablation) | MSE, Adv-MSE and decision-aware critic losses · Cliff World | policy return J(pi) | Helped |
| minimize squared loss on Q versus decision-aware loss in a two-armed bandit with linear critic | direct representation, tabular policy · Two-armed bandit (r1 = 2, r2 = 1) | convergence to optimal arm: squared loss fails when p_0 < 2/5 | Helped |
| estimate Q by Monte-Carlo samples instead of true dynamics, or use tabular instead of linear policy (ablation) | decision-aware actor-critic · Cliff World and Frozen Lake | policy return J(pi) | No clear effect |
Closest papers by meaning
- Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation (ICML 2024 · similarity 0.91)
- Actor-Critics Can Achieve Optimal Sample Efficiency (ICML 2025 · similarity 0.91)
- ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages (ICML 2024 · similarity 0.90)
- A Theoretical Justification for Asymmetric Actor-Critic Algorithms (ICML 2025 · similarity 0.89)
- Decoupled Policy Actor-Critic: Bridging Pessimism and Risk Awareness in Reinforcement Learning (AAAI 2025 · similarity 0.89)
- Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.