Research map / Reinforcement learning / Policy gradient and actor-critic
Anchor-Changing Regularized Natural Policy Gradient for Multi-Objective Reinforcement Learning
Ruida Zhou, Tao Liu, Dileep Kalathil, Panganamala Kumar, Chao Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ARNPG-EPD instead of NPG-PD | Tabular CMDP exact gradients · random CMDP | average optimality gap and constraint violation | Improved |
| Compare ARNPG-EPD to CRPO | Tabular CMDP exact gradients · random CMDP | optimality gap | Mixed |
| Use ARNPG-EPD instead of NPG-PD and CRPO with sampled gradients | Sample-based tabular CMDP · random CMDP | reward value | Improved |
| Use ARNPG-EPD instead of NPG-PD and CRPO | Acrobot-v1 · Acrobot-v1 CMDP | reward value | Improved |
| Increase InnerLoop iterations t_k from 1 to 5 or 10 (ablation) | ARNPG-EPD · Acrobot-v1 CMDP | stability and feasibility speed | Mixed |
| Use t_k=1 instead of larger t_k (ablation) | ARNPG-EPD · Acrobot-v1 CMDP | mean reward at iteration 300: 181.8 | Hurt |
| Replace NPG update with PPO (ARPPO-EPD) | ARNPG-EPD · Hopper-v3 | reward value | Improved |
| Increase InnerLoop iterations t_k (ablation) | ARNPG-IMD · sum-logarithmic two-objective MDP, exact gradients | average optimality gap | Improved |
| Use ARNPG-OMDA instead of MO-NPG | max-min two-objective MDP · exact gradients | optimality gap | Improved |
| Increase t_k above 1 (ablation) | sample-based ARNPG-IMD · sum-logarithmic MOMDP | scalarized objective | Mixed |
| Use sample-based ARNPG-OMDA instead of MO-NPG | max-min MOMDP · sample-based tabular | scalarized objective | Improved |
| Vary t_k in 1,2,5,10 (ablation) | ARNPG-IMD · Acrobot-v1 MOMDP | scalarized objective | No clear effect |
Closest papers by meaning
- Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm (NeurIPS 2024 · similarity 0.89)
- Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework (NeurIPS 2026 · similarity 0.88)
- Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes (AAAI 2024 · similarity 0.88)
- Policy Gradient Algorithms in Average-Reward Multichain MDPs (NeurIPS 2026 · similarity 0.88)
- Confident Natural Policy Gradient for Local Planning in $q_\pi$-realizable Constrained MDPs (NeurIPS 2024 · similarity 0.87)
- Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
