Research map / Reinforcement learning / Policy gradient and actor-critic
Representations for Stable Off-Policy Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Schur representation instead of singular vectors of the successor representation | TD(0) policy evaluation · Four-room domain, off-policy data | stability | Helped |
| Use an orthogonal Krylov basis instead of the Krylov basis (ablation) | TD(0) policy evaluation · Four-room domain, off-policy data | stability | Helped |
| Use the safe Laplacian representation, discarding eigenvectors above the stability threshold | TD(0) policy evaluation · Four-room domain, off-policy data | stability | Helped |
| Discard the top eigenvectors of the symmetrized transition matrix (ablation) | Safe Laplacian representation · Four-room domain, off-policy data | policy accuracy | Hurt |
| Use the Schur representation rather than singular-vector or spectral representations | Representation estimation from samples · Four-room domain | learnability | Hurt |
| Learn Schur features by predicting next-step target-network features with orthogonality enforcement | Neural-network representation learning · Four-room domain | stability | Helped |
| Learn Krylov features by predicting future rewards | Neural-network representation learning · Four-room domain | stability | Helped |
| Use singular-vector representations that minimize approximation error without preserving transition-matrix spectrum | TD(0) policy evaluation · Off-policy policy evaluation | stability | Hurt |
Closest papers by meaning
- Stable Offline Value Function Learning with Bisimulation-based Representations (ICML 2025 · similarity 0.90)
- Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism (ICLR 2022 · similarity 0.89)
- Representation Balancing Offline Model-based Reinforcement Learning (ICLR 2021 · similarity 0.89)
- Online Laplacian-Based Representation Learning in Reinforcement Learning (ICML 2025 · similarity 0.87)
- A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning (NeurIPS 2025 · similarity 0.87)
- Understanding and Diagnosing Deep Reinforcement Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.