Research map / Probabilistic and scientific modeling / Dynamical systems
Understanding Self-Predictive Learning for Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use faster-paced (near-optimal) prediction matrix P_t together with semi-gradient updates on Phi (ablation) | Self-predictive learning (single representation) · Random tabular MDPs (k=2) | cosine similarity between representation columns (collapse indicator) | Helped |
| Replace semi-gradient with full gradient on Phi (no stop-gradient on target) (ablation) | Self-predictive learning · Random tabular MDPs (k=2) | column collapse (cosine similarity) | Hurt |
| Corrupt the optimal predictor with zero-mean noise each iteration (ablation) | Self-predictive learning · Random tabular MDPs (k=2) | column collapse (cosine similarity) | Hurt |
| Use symmetric transition matrix P^pi (vs non-symmetric) | Self-predictive learning trace objective · 100 random tabular MDPs, exact ODE | trace objective improvement over time | Helped |
| Bidirectional (forward+backward) self-predictive learning instead of single representation (ablation) | Self-predictive learning on 3-state transition matrix example · Tabular MDP with asymmetric singular vectors | SVD trace objective ratio to top-k value | Helped |
| Add backward latent self-prediction to BYOL-RL (deep bidirectional self-predictive learning) | BYOL-RL on DMLab-30 · DMLab-30 multi-task | mean human normalized score | No clear effect |
| Add backward prediction to BYOL-RL | BYOL-RL on DMLab-30 per-game · DMLab-30 multi-task | per-game human normalized score gain: up to about 0.4 gain on some tasks | Mixed |
Closest papers by meaning
- Bridging State and History Representations: Understanding Self-Predictive RL (ICLR 2024 · similarity 0.90)
- Reinforcement Learning Under Latent Dynamics: Toward Statistical and Algorithmic Modularity (NeurIPS 2024 · similarity 0.87)
- Towards Principled Representation Learning from Videos for Reinforcement Learning (ICLR 2024 · similarity 0.87)
- Predictive Coding Enhances Meta-RL To Achieve Interpretable Bayes-Optimal Belief Representation Under Partial Observability (NeurIPS 2025 · similarity 0.86)
- Online Laplacian-Based Representation Learning in Reinforcement Learning (ICML 2025 · similarity 0.86)
- Diffusion Spectral Representation for Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.