AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning

Yuexin Bian, Jie Feng, Tao Wang, Yijiang Li, Sicun Gao, Yuanyuan Shi

ICML 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Off-policy actor-critic with expressive policies · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace Gaussian actor with discretized categorical actor and add regularized residual network (RN-D)PPO · Gym locomotion and ManiSkillsample efficiency: 1.3--1.9x fewer interaction stepsImproved
add regularized residual pre-LayerNorm network to Gaussian actor (ablation)MLP-C · Gym locomotion and ManiSkill-Statereturn / successImproved
discretize actions with categorical actor on a standard MLP (ablation)MLP-C · Gym locomotion and ManiSkill-Statereturn / successImproved
pair categorical actor with regularized network on vision tasksMLP-C · ManiSkill RGBsuccess rateMixed
swap categorical likelihood for Gaussian squared-error loss on discretized actor (ablation)RN-D · Gym locomotion / StackCubereturnHurt
use discretized categorical actor instead of GaussianRN-C / MLP-C · MuJoCo taskspolicy-gradient varianceImproved
add residual connections and layer normalization to actor (ablation)MLP actor · Gym locomotiongradient SNRImproved
increase actor hidden width from 64 to 512 (ablation)RN-D · Humanoid-v4 and StackCube-v1convergence speedMixed
increase number of action bins beyond about 100 (ablation)RN-D · StackCube-v1success rateHurt
increase action bins from 11 to 1001 (ablation)RN-D · Humanoid-v4wall-clock time (h): 0.76 -> 1.95Hurt
use discretized instead of continuous actor with standard MLPMLP-C · Gym locomotionwall-clock time (h): 1.65 -> 1.56Improved
use discretized actor with regularized networkRN-C · Gym locomotionSPS: 811.1 -> 873.8Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy