Research map / Reinforcement learning / Policy gradient and actor-critic
LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration
Ruiyu Qiu, Rui Wang, Guanghui Yang, Xiang Li, Zhijiang Shao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace generic CVXPY solvers with Dykstra's projection for the gradient projection (ablation) | LPPG-RL · Nav2D-1G | runtime (ms): 18.30 -> 0.93 (OSQP to Dykstra) | Improved |
| replace generic CVXPY solvers with Dykstra's projection (ablation) | LPPG-RL · Nav2D-100G | runtime (ms): OSQP 2907.73 vs Dykstra 2983.20 | No clear effect |
| replace generic CVXPY solvers with Dykstra's projection (ablation) | LPPG-RL · Nav2D-nG | final policy performance | No clear effect |
| add Subproblem Exploration (uniform rollout scheduler over priority levels) (ablation) | LPPG-PPO · Nav2D-2G | lexicographic priority satisfied (K3 vs K4 return): K3 199 not completed -> 587 completed | Improved |
| add Subproblem Exploration (ablation) | LPPG-PPO · Nav2D-1G | subtask completion: obstacle avoidance K2 -12 not completed -> 0 completed | Improved |
| use projected gradient with SE instead of threshold-based LPA | LMORL baselines · Nav2D-2G | subtask completion | Improved |
| use projected gradient instead of Lagrangian-multiplier LPPO | LMORL baselines · Nav2D-2G | subtask completion | Improved |
| LPPG-PPO gives lower K3 return than baselines on Nav2D-1G while satisfying constraints | LPPG-PPO · Nav2D-1G | K3 return: LPPO 740 vs Ours 427 | Hurt |
Closest papers by meaning
- Multi-objective Linear Reinforcement Learning with Lexicographic Rewards (ICML 2025 · similarity 0.89)
- Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF (ICML 2025 · similarity 0.89)
- Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs (NeurIPS 2025 · similarity 0.89)
- LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.88)
- On Generalization Across Environments In Multi-Objective Reinforcement Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
