Research map / Reinforcement learning / Policy gradient and actor-critic
Iterative Amortized Policy Optimization
Joseph Marino, Alexandre Piché, Alessandro Davide Ialongo, Yisong Yue
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace direct amortization with iterative amortization (5 iterations) | SAC policy · MuJoCo continuous control | cumulative reward | Improved |
| increase policy optimization iterations per step during training from 1 to 2 to 5 (ablation) | Iterative amortization · HalfCheetah-v2 | performance and amortization gap | Improved |
| add normalizing-flow multi-modal policy to direct amortization (ablation) | Direct amortization · Hopper-v2, Walker2d-v2 | cumulative reward | Improved |
| add normalizing-flow multi-modal policy to direct amortization (ablation) | Direct amortization · HalfCheetah-v2 | cumulative reward | No clear effect |
| run 5 additional amortized iterations at evaluation (ablation) | Iterative amortization trained with 5 iterations · MuJoCo amortization gap | estimated amortization gap | Improved |
| deploy more fully optimized policy at evaluation (ablation) | Iterative amortization · MuJoCo environments | evaluation return | No clear effect |
| compare iterative amortization with Adam and CEM optimizers | Policy optimization · Variational objective, 100 states per seed | iterations to reach objective | Improved |
| transfer iterative optimizer from model-free to model-based value estimates without retraining | Iterative amortization · HalfCheetah-v2 | cumulative reward | Improved |
| iterative vs direct amortization on simple pendulum tasks | SAC policy · InvertedPendulum-v2, InvertedDoublePendulum-v2 | cumulative reward | No clear effect |
Closest papers by meaning
- Action abstractions for amortized sampling (ICLR 2025 · similarity 0.86)
- Wasserstein Policy Optimization (ICML 2025 · similarity 0.86)
- Learning Intractable Multimodal Policies with Reparameterization and Diversity Regularization (NeurIPS 2025 · similarity 0.86)
- Amortized Active Causal Induction with Deep Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization (ICML 2024 · similarity 0.85)
- Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
