Research map / Reinforcement learning / Policy gradient and actor-critic
Experience Selection in Deep Reinforcement Learning for Control
Tim de Bruin, Jens Kober, Karl Tuyls, Robert Babuška
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| sample with PER instead of uniform sampling | TDE(1.0) retention · Pendulum swing-up with noise | mean absolute noise (position) in mini-batch: 1.713·10−2 -> 1.846·10−2 | Hurt |
| sample with PER instead of uniform sampling | Expl(1.0) retention · Pendulum swing-up with noise | mean absolute noise (position) in mini-batch: 1.584 ·10−2 -> 1.654 ·10−2 | Hurt |
| retain all experiences (FULL DB) with PER instead of FIFO with uniform sampling | DDPG-style actor-critic · Pendulum swing-up | normalized mean reward | Improved |
| retain all experiences (FULL DB) with PER instead of FIFO with uniform sampling | DDPG-style actor-critic · Magman | normalized mean reward | Hurt |
| add PER sampling when sampling from diverse experiences | FIFO / Resv retention · Magman | final performance | Hurt |
| add PER sampling | FIFO retention · Magman | learning speed | Improved |
| add PER sampling | retention strategies · Pendulum swing-up | learning speed | Improved |
| add PER sampling under sensor and actuator noise | retention strategies · Pendulum and Magman with noise 0.02 | final performance | Hurt |
| use reservoir retention instead of storing all experiences | Resv[Uniform] with 10^4 buffer · Pendulum swing-up | normalized mean reward | No clear effect |
| use a small FIFO buffer of recent experiences | FIFO[Uniform] · Magman at 50 Hz | final performance | Improved |
| use FIFO retention at higher sampling frequency (200 Hz) | FIFO[Uniform] · Magman | final performance | Hurt |
| add sensor and actuator noise | experience diversity need · Pendulum and Magman | need for diverse data | No clear effect |
Closest papers by meaning
- Revisiting Fundamentals of Experience Replay (ICML 2020 · similarity 0.92)
- Learning to Sample with Local and Global Contexts in Experience Replay Buffer (ICLR 2021 · similarity 0.92)
- Selective Experience Replay for Lifelong Learning (AAAI 2018 · similarity 0.92)
- Remember and Forget for Experience Replay (ICML 2019 · similarity 0.91)
- Attentive Experience Replay (AAAI 2020 · similarity 0.91)
- Variance Reduction via Resampling and Experience Replay (AAAI 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
