Research map / Reinforcement learning / Policy gradient and actor-critic
Model-Augmented Actor-Critic: Backpropagating through Paths
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add test-time MPC | MAAC · AntEnv | return: 3.06\text{e}3 \pm 1.45\text{e}3 -> 3.97\text{e}3 \pm 1.48\text{e}3 | Helped |
| Add test-time MPC | MAAC · HalfCheetahEnv | return: 1.07\text{e}4 \pm 253 -> 1.09\text{e}4 \pm 94.5 | Helped |
| Add test-time MPC | MAAC · HopperEnv | return: 2.77\text{e}3 \pm 3.31 -> 2.8\text{e}3 \pm 11 | Helped |
| Add test-time MPC | MAAC · Walker2dEnv | return: 1.61\text{e}3 \pm 404 -> 1.76\text{e}3 \pm 78 | Helped |
| Add model-based pathwise policy gradients instead of setting H=0 (ablation) | MAAC · MuJoCo continuous-control benchmarks | return | Helped |
| Use STEVE targets to train the critic (ablation) | MAAC · MuJoCo continuous-control benchmarks | training stability | Helped |
| Use multiple pathwise-derivative samples instead of a single sample (ablation) | MAAC · Ant | gradient-estimate accuracy | Helped |
| Use dynamics-generated data to train the policy and Q-function (ablation) | MAAC · MuJoCo continuous-control benchmarks | return | Helped |
| Add an entropy bonus to the optimization objective (ablation) | MAAC · MuJoCo continuous-control benchmarks | return | Helped |
| Use a larger batch to compensate for a single pathwise-derivative sample (ablation) | MAAC · MuJoCo continuous-control benchmarks | performance | No clear effect |
| Add test-time MPC | MAAC · Complex domains | return | Helped |
Closest papers by meaning
- Actor-Critics Can Achieve Optimal Sample Efficiency (ICML 2025 · similarity 0.90)
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.90)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.89)
- Wasserstein Policy Optimization (ICML 2025 · similarity 0.89)
- A Theoretical Justification for Asymmetric Actor-Critic Algorithms (ICML 2025 · similarity 0.89)
- Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.