AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

FlowPG: Action-constrained Policy Gradient with Normalizing Flows

Janaka Chathuranga Brahmanage, Jiajing Ling, Akshat Kumar

NeurIPS 2023 · Reinforcement learning › Policy gradient and actor-critic · approach: Constrained policy optimization · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use FlowPG's normalizing flow rather than WGAN (ablation)Reacher action generation · Reachervalid-action accuracy: 98% -> 99.98%Improved
Use FlowPG's normalizing flow rather than VAE (ablation)Reacher action generation · Reachervalid-action accuracy: 83% -> 99.98%Improved
Use HMC rather than rejection sampling to generate feasible training actions (ablation)FlowPG flow training · Reachervalid-action sampling success rate: 3.93% -> 100%Improved
Use HMC rather than rejection sampling to generate feasible training actions (ablation)FlowPG flow training · Half Cheetahvalid-action sampling success rate: 4.7% -> 100%Improved
Replace the standard Gaussian latent prior with a mollified uniform prior (ablation)FlowPG · Action-constrained continuous controlconstraint violationsImproved
Use the learned flow mapping instead of QP projection for policy actionsDDPG policy in action-constrained tasks · Reacher and BSSaverage returnImproved
Use the learned flow mapping instead of Frank-Wolfe policy updates requiring QP solutionsDDPG policy in action-constrained tasks · Continuous control taskstraining speed: 2~3 timesImproved
Use FlowPG rather than DDPG+ProjectionDDPG on action-constrained tasks · Reacheraverage returnImproved
Use FlowPG rather than NFWPODDPG on action-constrained tasks · Half Cheetah, Hopper, and Walker2daverage returnNo clear effect
Use FlowPG rather than DDPG+Projection or NFWPODDPG on action-constrained tasks · All environments except Walker2dcumulative constraint violations: an order of magnitudeImproved
Use FlowPG rather than the projection-based baselinesDDPG on action-constrained tasks · All environments except Hopper and Walker2daverage magnitude of constraint violationsImproved
Use FlowPG rather than DDPG+ProjectionDDPG on action-constrained tasks · Continuous control tasksaverage returnImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy