Research map / Reinforcement learning / Policy gradient and actor-critic
Gamma-Nets: Generalizing Value Estimation over Timescale
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add timescale input to the value estimator | Gamma-net · Robot arm, γ=0.9 | cumulative absolute error: 1124 -> 1025 | Helped |
| Add timescale input to the value estimator | Gamma-net · Robot arm, γ=0.9666 | cumulative absolute error: 822 -> 602 | Helped |
| Add timescale input to the value estimator | Gamma-net · Robot arm, γ=0.98333 | cumulative absolute error: 440 -> 379 | Helped |
| Add timescale input to the value estimator | Gamma-net · Robot arm, γ=0.99 | cumulative absolute error: 253 -> 273 | Hurt |
| Use both γ and τ rather than γ alone as timescale inputs (ablation) | Gamma-net · Square wave | normalized MSE | Helped |
| Use both γ and τ rather than γ alone as timescale inputs (ablation) | Gamma-net · Atari games | MSE | Helped |
| Draw training timescales from both γ and τ scales instead of γ alone (ablation) | Gamma-net · Atari games | MSE | Helped |
| Draw training timescales from both γ and τ scales instead of τ alone (ablation) | Gamma-net · Atari games | MSE | Mixed |
| Scale the loss by (1-γ) (ablation) | Gamma-net · Centipede@25M | MSE | Helped |
| Scale the loss by (1-γ) (ablation) | Gamma-net · Atari games | MSE | Mixed |
| Use linear rather than ReLU activation in the embedding (ablation) | Gamma-net embedding · Centipede@25M | MSE | Helped |
| Replace direct concatenation with a learned linear embedding (ablation) | Gamma-net · Centipede@25M | MSE | No clear effect |
Closest papers by meaning
- Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction (AAAI 2021 · similarity 0.90)
- Two-Timescale Networks for Nonlinear Value Function Approximation (ICLR 2019 · similarity 0.89)
- Rethinking Value Function Learning for Generalization in Reinforcement Learning (NeurIPS 2022 · similarity 0.89)
- Understanding End-to-End Model-Based Reinforcement Learning Methods as Implicit Parameterization (NeurIPS 2021 · similarity 0.88)
- Adaptive Exploration for Data-Efficient General Value Function Evaluations (NeurIPS 2024 · similarity 0.87)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.