Research map / Reinforcement learning / Visual and representation RL
A State Representation for Diminishing Rewards
Ted Moskovitz, Samo Hromadka, Ahmed Touati, Diana L Borsa, Maneesh Sahani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use lambda-representation with lambda = true lambda (0.5) instead of SR (lambda = 1) (ablation) | value estimation (DP, TD, lambda-F TD) · FourRooms policy evaluation | value estimate accuracy | Improved |
| use Q_lambda-learning with correct lambda instead of standard Q-learning (lambda = 1) (ablation) | Q-learning · TwoRooms (NeuroNav) | return | Improved |
| use lambda-R critic with correct lambda in A2C (ablation) | A2C · TwoRooms (NeuroNav) | return | Improved |
| set lambda too high (1.0) for GPE+GPI (ablation) | tabular GPE+GPI · FourRooms | average return | Hurt |
| set lambda too low (0.0) for GPE+GPI (ablation) | tabular GPE+GPI · FourRooms | average return | Hurt |
| use correct lambda = 0.5 in lambda-F GPI (ablation) | pixel-based GPE+GPI · partially observed FourRooms | return | Improved |
| lower lambda in lambda-R learning (ablation) | dynamic programming · FourRooms | Bellman error convergence speed | Improved |
| learn lambda online alongside the lambda-O (ablation) | lambda-O trained via FB · FourRooms | performance | Improved |
| use an incorrect value of lambda (ablation) | tabular tasks · tabular foraging/navigation | performance | Hurt |
Closest papers by meaning
- Reward-Aware Proto-Representations in Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- A Distributional Analogue to the Successor Representation (ICML 2024 · similarity 0.87)
- Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (NeurIPS 2024 · similarity 0.87)
- Hindsight PRIORs for Reward Learning from Human Preferences (ICLR 2024 · similarity 0.87)
- Proto Successor Measure: Representing the Behavior Space of an RL Agent (ICML 2025 · similarity 0.86)
- Bridging State and History Representations: Understanding Self-Predictive RL (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
