Research map / Reinforcement learning / Visual and representation RL
The Utility of Sparse Representations for Control in Reinforcement Learning
Vincent Liu, Raksha Kumaraswamy, Lei Le, Martha White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a learned sparse representation rather than a dense neural-network representation for incremental Sarsa control | Sarsa(0) · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Improved |
| Use tile coding rather than a dense neural-network representation | Sarsa(0) · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Improved |
| Add Set KL instead of exact KL to the ReLU activation regularizer (ablation) | ReLU distributionally regularized network · Mountain Car, Puddle World, Acrobot, Catcher | instance sparsity | Improved |
| Add Set KL instead of exact KL to the ReLU activation regularizer (ablation) | Sarsa(0) with learned representations · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Improved |
| Use ReLU with an exponential target distribution rather than Sigmoid with a Bernoulli target (ablation) | Distributionally regularized neural representation · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Improved |
| Use ReLU with an exponential target distribution rather than Sigmoid with a Bernoulli target (ablation) | Distributionally regularized neural representation · Mountain Car, Puddle World | activation overlap: 16.8 vs 256.0; 8.8 vs 256.0 | Improved |
| Add an activation ℓ2 regularizer (ablation) | Sarsa(0) with learned representations · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Mixed |
| Add an activation ℓ1 regularizer (ablation) | Sarsa(0) with learned representations · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Hurt |
| Use top-k activation truncation (ablation) | Neural representation for Sarsa(0) · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Hurt |
| Use winner-take-all activation selection across instances (ablation) | Neural representation for Sarsa(0) · Mountain Car, Puddle World, Acrobot, Catcher | control performance | Hurt |
| Add k-sparse truncation to the ReLU+Set-KL network (ablation) | ReLU+SKL representation · Mountain Car, Puddle World, Acrobot, Catcher | control performance | No clear effect |
| Use sparse rather than dense representations | Sarsa(0) action-value learning · Puddle World | activation overlap: 8.8 vs 54.0 | Improved |
Closest papers by meaning
- The State of Sparse Training in Deep Reinforcement Learning (ICML 2022 · similarity 0.91)
- Learning Actionable Representations with Goal Conditioned Policies (ICLR 2019 · similarity 0.88)
- Fuzzy Tiling Activations: A Simple Approach to Learning Sparse Representations Online (ICLR 2021 · similarity 0.88)
- Sparse Graphical Memory for Robust Planning (NeurIPS 2020 · similarity 0.87)
- Bootstrapped Representations in Reinforcement Learning (ICML 2023 · similarity 0.87)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
