Research map / Reinforcement learning / Policy gradient and actor-critic
Composing Entropic Policies using Divergence Correction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Correct the composed action value using the learned future policy-divergence term | Composition of max-ent policies · Tabular 8x8 grid world, tricky task | log regret | Helped |
| Use compositional optimism, which combines component action values without a divergence correction | Composition of max-ent policies · Tabular 8x8 grid world, Left-Right | log regret | Hurt |
| Use max-ent GPI to select actions from the best component-task value | Composition of max-ent policies · Tabular 8x8 grid world, Left-Up | log regret | Hurt |
| Use compositional optimism when the component policies are compatible | Composition of max-ent policies · Tabular 8x8 grid world, Left-Up | log regret | Helped |
| Correct the composed action value using learned future divergence | Composition of max-ent policies · Point mass, tricky task | returns | Helped |
| Correct the composed action value using learned future divergence | Composition of max-ent policies · Planar manipulator, tricky task | returns | Helped |
| Correct the composed action value using learned future divergence | Composition of max-ent policies · Jumping ball, tricky task | returns | Helped |
| Correct the composed action value using learned future divergence | Composition of max-ent policies · Ant, tricky task | returns | Helped |
| Replace the task-conditional correction with a correction estimated from b=1/2 (ablation) | Divergence Correction (DC) · Point mass, tricky task | returns | Hurt |
| Add a max-ent GPI lower-bound safeguard to DC-Cheap (ablation) | DC-Cheap · Point mass, tricky task | returns | Helped |
| Replace the task-conditional correction with a correction estimated from b=1/2 (ablation) | Divergence Correction (DC) · Planar manipulator, tricky task | returns | No clear effect |
| Replace the task-conditional correction with a correction estimated from b=1/2 (ablation) | Divergence Correction (DC) · Ant, tricky task | returns | No clear effect |
Closest papers by meaning
- Composing Value Functions in Reinforcement Learning (ICML 2019 · similarity 0.89)
- Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning (ICLR 2024 · similarity 0.86)
- Mirror descent actor-critic methods for entropy regularised MDPs in general spaces: stability and convergence (NeurIPS 2026 · similarity 0.86)
- When Maximum Entropy Misleads Policy Optimization (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.