Research map / Reinforcement learning / Policy gradient and actor-critic
Generalization and Exploration via Randomized Value Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| randomized least-squares value function sampling (RLSVI) instead of dithering exploration | LSVI with linear value functions · 50-state chain with coherent random basis (K=10) | episodes to learn optimal policy: roughly 500 episodes vs at least 10^15 for any dithering strategy (theoretical) | Helped |
| RLSVI exploration instead of annealed epsilon-greedy | LSVI on Tetris · Tetris with 22 benchmark features | final score: ≃3500 vs ≃4500 reported for LSVI tuned epsilon (text says RLSVI is worse here, so check) | Helped |
| RLSVI exploration vs LSVI with tuned epsilon-greedy schedule | Tetris · Tetris with 22 benchmark features | final score | Mixed |
| RLSVI exploration vs Boltzmann exploration in LSVI | Recommendation system with linear basis · Recommendation engine (N=10, J=5) | cumulative regret | Helped |
| randomized value exploration vs epsilon-greedy in agnostic setting | RLSVI with misspecified basis · 20-state chain, K=20, noise rho | episodes until 10 rewards | Mixed |
| sigma too small in RLSVI (ablation) | RLSVI on stochastic domains · chain experiments parameter sweep | learning speed / exploration | Hurt |
| large sigma in RLSVI (ablation) | RLSVI · chain experiments parameter sweep | learning speed | Hurt |
| RLSVI with respect to lambda and sigma sensitivity (ablation) | RLSVI · chain N=50, K=10 | cumulative regret | No clear effect |
| adding backward value propagation (RLSVI) vs linear contextual bandit without it | Recommendation system · Recommendation engine, long horizon | cumulative reward vs benchmarks | Helped |
Closest papers by meaning
- Random Latent Exploration for Deep Reinforcement Learning (ICML 2024 · similarity 0.91)
- Concurrent Reinforcement Learning with Aggregated States via Randomized Least Squares Value Iteration (ICML 2025 · similarity 0.89)
- Randomized Exploration for Reinforcement Learning with Multinomial Logistic Function Approximation (NeurIPS 2024 · similarity 0.89)
- Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning (ICLR 2024 · similarity 0.88)
- Making RL with Preference-based Feedback Efficient via Randomization (ICLR 2024 · similarity 0.88)
- Exploration from a Primal-Dual Lens: Value-Incentivized Actor-Critic Methods for Sample-Efficient Online RL (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.