AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximation

Junhong Shen, Lin F. Yang

AAAI 2021 · Reinforcement learning › Policy gradient and actor-critic · approach: Off-policy actor-critic with expressive policies · 23 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add the soft nearest-neighbor update moduleTD3 · Antaverage return: 3425.07 -> 4727.44Improved
Add the soft nearest-neighbor update moduleDDPG · Antaverage return: 1025.82 -> 1115.00Improved
Add the soft nearest-neighbor update moduleTD3 · Hopperaverage return: 2708.25 -> 3704.40Improved
Add the soft nearest-neighbor update moduleDDPG · Hopperaverage return: 2029.65 -> 3202.64Improved
Add the soft nearest-neighbor update moduleTD3 · Walker2daverage return: 4260.23 -> 5170.20Improved
Add the soft nearest-neighbor update moduleDDPG · Walker2daverage return: 2633.86 -> 2917.25Improved
Add the soft nearest-neighbor update moduleTD3 · HalfCheetahaverage return: 7045.53 -> 8341.74Improved
Add the soft nearest-neighbor update moduleDDPG · HalfCheetahaverage return: 6385.43 -> 7001.52Improved
Use L2 distance for image similarity instead of a learned distance metric (ablation)NNAC · CartPole-v1 pixel inputlearning stabilityHurt
Use a learned distance metric for pixel input (ablation)NNAC · CartPole-v1 pixel inputsample efficiencyMixed
Use projected states rather than intrinsic state descriptors (ablation)NNAC · CartPole-v1convergence stepsNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy