Research map / Reinforcement learning / MDPs and planning
Non-parametric Policy Search with Limited Information Loss
Herke van Hoof, Gerhard Neumann, Jan Peters
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use KL bound epsilon 0.5 rather than epsilon 5 in NP-REPS (ablation) | NP-REPS with model · Low-dimensional swing-up | average reward learning curve: epsilon = 5 vs epsilon = 0.5 | Improved |
| use KL bound epsilon 0.5 rather than epsilon 0.05 (ablation) | NP-REPS with model · Low-dimensional swing-up | learning speed: epsilon = 0.05 vs epsilon = 0.5 | Improved |
| model transitions with learned model instead of sample-based (ablation) | NP-REPS · Low-dimensional swing-up (stochastic) | average reward | Improved |
| learned model instead of sample-based transitions (ablation) | NP-REPS · Reaching task (deterministic) | learning speed | Hurt |
| use non-parametric representation instead of fixed features (ablation) | REPS · Low-dimensional swing-up | average reward in later iterations | Improved |
| use Fourier features with 500 bases instead of sub-sampling (ablation) | kernel approximation · Swing-up approximation (500 bases) | average reward: sampled subset performs very badly at 500 bases | Improved |
| use regression-based sparsification instead of Fourier features (ablation) | kernel approximation · Swing-up approximation (100 bases) | learning speed: about 7% of training data | Improved |
| use Fourier approximation with 500 bases instead of exact kernel (ablation) | NP-REPS · Swing-up approximation | computation time: about twice as fast at a similar level of performance | Improved |
| use NP-REPS instead of on-policy value iteration | policy update · Low-dimensional swing-up | average reward | Improved |
| use NP-REPS instead of DDPG | image-based policy search · Image-based puddle world | return | Improved |
| use joint values instead of camera image representation | NP-REPS · Real-robot swing-up | learning performance | Improved |
Closest papers by meaning
- Regularized Policy Iteration with Nonparametric Function Spaces (JMLR 2016 · similarity 0.87)
- Batch Reinforcement Learning With a Nonparametric Off-Policy Policy Gradient (TPAMI 2021 · similarity 0.87)
- On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control (JMLR 2024 · similarity 0.87)
- Policy Gradient for Rectangular Robust Markov Decision Processes (NeurIPS 2023 · similarity 0.87)
- MetaCURL: Non-stationary Concave Utility Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Oracle-Efficient Reinforcement Learning for Max Value Ensembles (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
