AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

LOCU: Löwdin-Orthogonalized Constraint Updates for Multi-Constraint Policy Optimization

Joonyoung Lim, Younghwan Yoo

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Constrained policy optimization

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LOCU, which decouples interacting constraint gradients in CMDP policy optimization via symmetric Löwdin orthogonalization to obtain better-conditioned constraint update directions.

Problem

Multi-constraint policy optimization in constrained Markov decision processes, where interactions among constraint gradients produce ill-conditioned, numerically fragile update directions with cancellation or redundant overlap between constraint corrections.

Approach

Applies symmetric Löwdin orthogonalization to the constraint gradients so that constraint interactions are decoupled before forming the policy update, yielding well-conditioned constraint corrections.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply Löwdin orthogonalization with screening and Pareto recovery instead of QP-based gradient integration (SDAC actor) on goal-navigation task with heterogeneous constraintsSDAC(A) baseline · goal-navigation with Cassie-style stability constraintsreturn and constraint satisfactionImproved
use LOCU in multi-constraint narrow-feasible-region and strongly coupled settings versus baselinesmulti-constraint policy optimization · Bullet-Safety-Gym Circle v3 tightening/redundancyreturn drop and constraint violations: return drop around 21% vs 26-49% and 13-32% in baselinesImproved
use LOCU's orthogonal basis in a reduced QP solved with Clarabel instead of the full-dimensional QP (ablation)full QP with Clarabel · goal-navigation taskreturn and constraint satisfactionImproved
replace LOCU's active-set solver with Clarabel interior-point solver on the full QP (ablation)full QP baseline · goal-navigation taskrecovery performance: solver flagged 2 of approximately 24,900 updates as inaccurateMixed
add gradient shaping or plain Lagrangian multipliers (PPOL variants) in the multi-constraint settingPPO Lagrangian · Circle tasks v2 (BC, CC, DC)circular tracking and constraint satisfactionHurt
use PCRPO soft-switching primal method in coupled multi-constraint locomotionPCRPO · Cassie and Laikago with three constraintsconvergenceHurt
use LOCU versus SDAC(A) on the original locomotion tasksSDAC(A) · Cassie and Laikago, three constraintsreturn with constraint satisfactionNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy