Research map / Reinforcement learning / Policy gradient and actor-critic
Variance Reduction for Reinforcement Learning in Input-Driven Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the state-dependent baseline with an input-dependent baseline | TRPO · MuJoCo robotic locomotion (Walker2d, HalfCheetah, 7-DoF arm) | testing reward: 25\%--3$\times$ better | Helped |
| Replace the state-dependent baseline with an input-dependent baseline | A2C · Load balancing and bitrate adaptation | testing reward: 25--33\% | Helped |
| Replace the state-dependent baseline with an input-dependent baseline | PPO · MuJoCo robotic locomotion | testing reward: 42\%--3.5$\times$ | Helped |
| Condition the baseline on the input sequence | A2C · Two-server load balancing example | policy gradient variance: 50$\times$ lower | Helped |
| Condition the baseline on the input sequence | A2C · Two-server load balancing example | test reward: 33\% higher | Helped |
| Use an LSTM to model the input-dependent baseline instead of the proposed efficient baseline approach (ablation) | A2C · Load balancing | policy performance | No clear effect |
| Use an LSTM to model the input-dependent baseline instead of the proposed efficient baseline approach (ablation) | TRPO · Walker2d with wind | policy performance | No clear effect |
| Vary the GAE parameter across the tested values (ablation) | TRPO with input-dependent baseline · MuJoCo robotic locomotion | performance: within \pm 5\% | No clear effect |
| Use the MAML meta-baseline instead of 10 separate value networks (ablation) | TRPO with input-dependent baseline · MuJoCo robotic locomotion | testing reward | Helped |
| Use an input-dependent baseline with a POMDP observation that omits the current wind (ablation) | TRPO · Walker2d with wind, POMDP | policy performance | Helped |
Closest papers by meaning
- Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines (ICLR 2018 · similarity 0.91)
- The Mirage of Action-Dependent Baselines in Reinforcement Learning (ICML 2018 · similarity 0.90)
- Time-Efficient Reinforcement Learning with Stochastic Stateful Policies (ICLR 2024 · similarity 0.87)
- Diffusion Policy Policy Optimization (ICLR 2025 · similarity 0.87)
- Towards Global Optimality for Practical Average Reward Reinforcement Learning without Mixing Time Oracles (ICML 2024 · similarity 0.87)
- Policy Evaluation for Variance in Average Reward Reinforcement Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.