Research map / Reinforcement learning / Policy gradient and actor-critic

Augmented Lagrangian Risk-constrained Reinforcement Learning for Portfolio Optimization (Student Abstract)

AAAI 2025 · Reinforcement learning › Policy gradient and actor-critic · approach: Constrained policy optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies an augmented Lagrangian constrained DDPG variant with a Dirichlet actor policy and a VaR cost network to portfolio optimization, reporting zero risk-constraint violations in tests.

Problem

Portfolio optimization must satisfy investor and regulator risk constraints (VaR), but standard RL portfolio methods maximize return or Sharpe ratio without hard risk constraints.

Approach

Extends DDPG with a separate cost network predicting VaR, an augmented Lagrangian actor objective enforcing the risk constraint, and a Dirichlet action distribution so asset weights sum to one.

Key results

• Proposed ALM model had 0 constraint violations on the test set with 35.7% annual average return and Sharpe ratio 0.3369.
• DDPG with expected-return objective achieved the highest return (40.3%) but had 154 constraint violations.
• Ablation: removing the Lagrangian multiplier or the quadratic penalty reduced performance to 96% and 94% of the full model; removing both gave 89%.

Measured effects reported in the paper

Surprising findings

• The highest-return baseline (DDPG, 40.3%) violated the risk constraint 154 times, showing return maximization conflicts with constraint compliance.

Limitations

• Short student abstract with limited experimental detail: dataset, asset universe, and test period are not described.
• Only a single test comparison and small ablation are reported; no variance or significance information.
• Proposed model converges more slowly than DDPG and earns lower return than the unconstrained baseline.

Experimental setup

Metrics: annual average return constraint violation number Sharpe ratio

Baselines: DDPG PPO TRPO BPG

Builds on: DDPG augmented Lagrangian method TRPO PPO

Closest papers by meaning

• Reward Penalties on Augmented States for Solving Richly Constrained RL Effectively (AAAI 2024 · similarity 0.86)
• Risk-Conditioned Reinforcement Learning: A Generalized Approach for Adapting to Varying Risk Measures (AAAI 2024 · similarity 0.86)
• Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets (NeurIPS 2025 · similarity 0.86)
• Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents (NeurIPS 2025 · similarity 0.86)
• Autoregressive Policy Optimization for Constrained Allocation Tasks (NeurIPS 2024 · similarity 0.85)
• Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning (ICML 2024 · similarity 0.84)