Research map / Reinforcement learning / Policy gradient and actor-critic

Deep learning for continuous-time stochastic control with jumps

NeurIPS 2025 · Reinforcement learning › Policy gradient and actor-critic · approach: Regularized scalable deep value learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes two model-based actor-critic deep learning algorithms (GPI-PINN and GPI-CBU) that iteratively train a value network and a control network for finite-horizon continuous-time stochastic control with jumps, with GPI-CBU avoiding gradients, Hessians and jump expectations via an expectation-free continuous-time Bellman update.</tardl>

Problem

Solving high-dimensional finite-horizon continuous-time stochastic control problems with jump-diffusion dynamics and implicit (non-closed-form) optimal controls, where classical PIDE solvers and model-free RL are inefficient.

Approach

Combine generalized policy iteration with an actor-critic split of two networks; GPI-PINN minimizes HJB residuals using a finite-difference-type trick for second-order terms, while GPI-CBU regresses the value network onto an expectation-free continuous-time Bellman target built from the previous value network, avoiding gradients, Hessians and jump integrals.

Key results

• GPI-CBU achieves MAE_V of 0.0147 for a 50-dimensional LQR problem with jumps after 5000 epochs (Table, d=50, Lambda2=2)
• GPI-CBU has lower runtime than GPI-PINN, especially with jumps, where GPI-PINN is infeasible at 50 dimensions
• GPI-CBU matches Runge-Kutta reference solutions virtually indistinguishably for constant-coefficient consumption-investment with 25 or 50 assets

Measured effects reported in the paper

Surprising findings

• Negative scaling factors zeta for GPI-CBU caused exploding losses and failed to converge in all experiments

Limitations

• Requires known system dynamics; model-learning is needed for data-driven settings
• Not as stable as the residual-based GPI-PINN; GPI-CBU convergence depends on scaling factor zeta

Experimental setup

Benchmarks: LQR with jumps (analytical/Runge-Kutta reference) optimal consumption-investment with jumps (Runge-Kutta reference)

Metrics: MAE_V MAE_alpha runtime loss

Baselines: GPI-PINN DQN C51 DDPG SAC A2C/A3C PPO TRPO Runge-Kutta reference solution

Builds on: Deep Galerkin Method PINN residual-based adaptive distribution (RAD) Feynman-Kac formula Verification theorem

Closest papers by meaning

• Consensus Based Stochastic Optimal Control (ICML 2025 · similarity 0.87)
• A Temporal Difference Method for Stochastic Continuous Dynamics (NeurIPS 2025 · similarity 0.86)
• Online Control with Adversarial Disturbance for Continuous-time Linear Systems (NeurIPS 2024 · similarity 0.86)
• Universal Approximation Theorem of Deep Q-Networks (ICML 2025 · similarity 0.85)
• Stochastic Q-learning for Large Discrete Action Spaces (ICML 2024 · similarity 0.85)
• Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.85)