Research map / Reinforcement learning / Policy gradient and actor-critic
Context-dependent upper-confidence bounds for directed exploration
Raksha Kumaraswamy, Matthew Schlegel, Adam White, Martha White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use context-dependent variance (UCLS) instead of global variance (GV-UCB) in the confidence bound (ablation) | UCLS · River Swim | total reward | Improved |
| decrease confidence parameter p (more exploration) (ablation) | UCLS · River Swim | convergence rate | Hurt |
| vary p over 10^-5 to 10^-1 (ablation) | UCLS · River Swim | final policy quality | No clear effect |
| use UCLS instead of other control algorithms (UCBootstrap, LSPI-Rmax, RLSVI, DGPQ, epsilon-greedy) | UCLS · Sparse Mountain Car, Puddle World, River Swim | final performance | Improved |
| use UCLS instead of UCBootstrap | UCLS · Puddle World | learning speed | No clear effect |
| use UCLS instead of DGPQ | UCLS · River Swim | initial learning speed | Hurt |
Closest papers by meaning
- Exploration in Structured Reinforcement Learning (NeurIPS 2018 · similarity 0.90)
- Tightening Exploration in Upper Confidence Reinforcement Learning (ICML 2020 · similarity 0.89)
- Beyond Optimism: Exploration With Partially Observable Rewards (NeurIPS 2024 · similarity 0.88)
- Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.88)
- Parameter Exploration for RLVR via Variational Learning (NeurIPS 2026 · similarity 0.88)
- Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
