Research map / Reinforcement learning / Visual and representation RL
Hyperparameters in Reinforcement Learning and How To Tune Them
Theresa Eimer, Marius Lindauer, Roberta Raileanu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HPO rather than manual hyperparameter sweeps | RL agents · Across tested RL environments | compute budget for comparable or better configurations: less than 1/12 of the budget | Improved |
| Tune the full search space rather than only learning rate (ablation) | PPO on Acrobot · Acrobot | negative evaluation reward: 71 ± 1 -> 71 ± 3 | No clear effect |
| Expand the search space from learning rate only to full (ablation) | SAC on Pendulum · Pendulum | negative evaluation reward: 71 ± 12 -> 112 ± 24 | Hurt |
| Expand the search space from learning rate only to full (ablation) | PPO on Acrobot · Acrobot | negative evaluation reward: 94 ± 22 -> 305 ± 186 | Hurt |
| Expand the search space from learning rate only to full (ablation) | SAC on Pendulum · Pendulum | negative evaluation reward: 207 ± 126 -> 78 ± 19 | Improved |
| Increase tuning seeds from 1 to 5 (ablation) | DEHB · Acrobot | test reward: 341.3±183.1 -> 465.1±24.6 | Improved |
| Increase tuning seeds from 1 to 5 (ablation) | DEHB · Pendulum | test reward: 150.5±13.4 -> 350.2 ± 418.2 | Mixed |
| Use separate test seeds rather than judging only by tuning-seed incumbents | HPO-selected RL configurations · Acrobot and Pendulum | performance generalization across seeds | Hurt |
| Use DEHB instead of hand-tuned baseline configurations | PPO on Brax · Brax | mean rank: 1.7 -> 1.3 (16 runs); 1.3 -> 1.0 (64 runs) | Improved |
| Use DEHB instead of hand-tuned baseline configurations | IDAAC on Procgen · Procgen | mean rank: 2 -> 1.7 (16 runs); 1.3 -> 1.0 (64 runs) | Improved |
| Increase RS tuning budget | RL agents on Brax and Procgen · Brax and Procgen | mean rank: 2.3 -> 3.3 (Brax); 2.7 -> 3 (Procgen) | Hurt |
| Tune across more seeds, beyond 3 (ablation) | HPO methods · Acrobot and Pendulum | test performance | Mixed |
Closest papers by meaning
- A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Towards General-Purpose Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Adaptive $Q$-Network: On-the-fly Target Selection for Deep Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learning (ICML 2025 · similarity 0.88)
- Parameter Exploration for RLVR via Variational Learning (NeurIPS 2026 · similarity 0.88)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
