Research map / Reinforcement learning / Policy gradient and actor-critic
On the role of overparameterization in off-policy Temporal Difference learning with linear function approximation
Valentin Thomas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Violate asymptotic well-connected property (4 rooms instead of random graph) | Spiked MP spectral prediction of TD operator · Relative error on lambda+_min | relative error: 0.3% -> 2% | Hurt |
| Non-uniform state sampling Xi (max/min xi = 2000) instead of uniform | Extreme eigenvalue prediction on 4 rooms · 4 rooms n=256 | match of lambda+_min and lambda_max | No clear effect |
| Move overparameterization ratio p/n far from 1 (rho much greater or less than 1) | TD optimization on 4 rooms · Bellman error vs iterations | optimization speed | Improved |
Closest papers by meaning
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.88)
- An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks (ICML 2024 · similarity 0.87)
- Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2026 · similarity 0.86)
- Revisiting a Design Choice in Gradient Temporal Difference Learning (ICLR 2025 · similarity 0.86)
- A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning (NeurIPS 2025 · similarity 0.86)
- Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
