AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Anchor-Changing Regularized Natural Policy Gradient for Multi-Objective Reinforcement Learning

Ruida Zhou, Tao Liu, Dileep Kalathil, Panganamala Kumar, Chao Tian

NeurIPS 2022 · Reinforcement learning › Policy gradient and actor-critic · approach: Variance-reduced policy gradients · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use ARNPG-EPD instead of NPG-PDTabular CMDP exact gradients · random CMDPaverage optimality gap and constraint violationImproved
Compare ARNPG-EPD to CRPOTabular CMDP exact gradients · random CMDPoptimality gapMixed
Use ARNPG-EPD instead of NPG-PD and CRPO with sampled gradientsSample-based tabular CMDP · random CMDPreward valueImproved
Use ARNPG-EPD instead of NPG-PD and CRPOAcrobot-v1 · Acrobot-v1 CMDPreward valueImproved
Increase InnerLoop iterations t_k from 1 to 5 or 10 (ablation)ARNPG-EPD · Acrobot-v1 CMDPstability and feasibility speedMixed
Use t_k=1 instead of larger t_k (ablation)ARNPG-EPD · Acrobot-v1 CMDPmean reward at iteration 300: 181.8Hurt
Replace NPG update with PPO (ARPPO-EPD)ARNPG-EPD · Hopper-v3reward valueImproved
Increase InnerLoop iterations t_k (ablation)ARNPG-IMD · sum-logarithmic two-objective MDP, exact gradientsaverage optimality gapImproved
Use ARNPG-OMDA instead of MO-NPGmax-min two-objective MDP · exact gradientsoptimality gapImproved
Increase t_k above 1 (ablation)sample-based ARNPG-IMD · sum-logarithmic MOMDPscalarized objectiveMixed
Use sample-based ARNPG-OMDA instead of MO-NPGmax-min MOMDP · sample-based tabularscalarized objectiveImproved
Vary t_k in 1,2,5,10 (ablation)ARNPG-IMD · Acrobot-v1 MOMDPscalarized objectiveNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy