Research map / Reinforcement learning / Policy gradient and actor-critic
Latent State Marginalization as a Low-cost Approach for Improving Exploration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SMAC instead of L-SAC with a world model | RSSM world model actor-critic · DMC noisy pixel observations, Gaussian perturbation (Finger, Hopper, Reacher) | episodic reward: Finger: 912 -> 959 (small), 880 -> 924 (large); Hopper: 571 -> 731 (small), 516 -> 702 (large); Reacher: 869 -> 928 (sma | Helped |
| Use SMAC instead of L-SAC with a world model | RSSM world model actor-critic · DMC noisy pixel observations, sensor missing (Finger, Hopper, Reacher) | episodic reward: Finger: 933 -> 955 (small), 921 -> 921 (large); Hopper: 721 -> 872 (small), 703 -> 866 (large); Reacher: 883 -> 937 (sma | Helped |
| Add latent-variable policies and marginalize the latent state | SAC on state-based continuous control · DMC state-based tasks | sample efficiency | Helped |
| Use SMAC rather than Latent-SAC | RSSM world model actor-critic · DMC pixel-based tasks | expected return | Helped |
| Use MLMC variance-reduced updates instead of the nested estimator alone (ablation) | SMAC latent-variable policy · quadruped_escape | training robustness | Helped |
| Increase the number of latent-state particles (ablation) | SMAC · quadruped_walk | training performance | Helped |
| Increase the number of particles used in estimation (ablation) | SMAC · DMC state-based tasks | training performance | Mixed |
| Use the naive upper-bound entropy estimator instead of the lower-bound nested estimator (ablation) | Latent-variable MaxEnt RL · Finger spin | policy learning: 10^{18} | Hurt |
| Add MaxEnt policy optimization to Dreamer (ablation) | Dreamer · continuous control | sample efficiency | Hurt |
| Add Russian Roulette estimation to the MLMC estimator (ablation) | SMAC · RL experiments | performance | No clear effect |
| Use a latent-variable policy rather than a single Gaussian policy (ablation) | MaxEnt policy for a one-step multimodal reward · Four-mode reward distribution | number of reward modes captured: four modes | Helped |
| Use SMAC rather than Latent-SAC | RSSM world model actor-critic · Pixel-based DMC training speed | frames per second: 63 -> 60 | Hurt |
Closest papers by meaning
- Reinforcement Learning Under Latent Dynamics: Toward Statistical and Algorithmic Modularity (NeurIPS 2024 · similarity 0.89)
- Generative Actor-Critic with Soft Bridge Policies (NeurIPS 2026 · similarity 0.89)
- Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning (ICML 2024 · similarity 0.89)
- Maximum Entropy Reinforcement Learning with Diffusion Policy (ICML 2025 · similarity 0.88)
- Global Optimality for Constrained Exploration via Penalty Regularization (NeurIPS 2026 · similarity 0.87)
- S$2$AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.