Research map / Reinforcement learning / Visual and representation RL
Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning
Nilaksh, Antoine Clavaud, Mathieu Reymond, Francois Rivest, Sarath Chandar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SPR auxiliary loss | QRC(lambda) · Atari 26 (40M frames, human normalized) | IQM return: 0.03 -> 0.61 | Improved |
| add orthogonal SPR gradient projection (orth) (ablation) | QRC(lambda)+SPR · Atari 26 (40M frames, human normalized) | IQM return: 0.61 -> 0.62 | Improved |
| add SPR auxiliary loss | streaming DQN · Atari 26 (40M frames, human normalized) | IQM return: 0.12 -> 0.22 | Improved |
| add naive SPR | Stream Q(lambda) · Atari 26 (40M frames, human normalized) | IQM return: 0.23 -> 0.13 | Hurt |
| add orth2 (orthogonalize SPR against ObGD update) (ablation) | Stream Q(lambda)+SPR+orth · Atari 26 (40M frames, human normalized) | IQM return: 0.12 -> 0.37 | Improved |
| add SPR with orth2 | Stream Q(lambda) · Atari 26 (100M frames, human normalized) | IQM return: 0.44 -> 0.56 | Improved |
| add SPR | streaming DQN · MinAtar (5M frames) | IQM return: 11.96 -> 32.60 | Improved |
| add SPR with orth | QRC(lambda) · Octax (20M frames) | IQM return: 12.52 -> 23.16 | Improved |
| add SPR loss | QRC(lambda) encoder · Atari 26 (40M frames) effective rank | effective rank: 42.7 -> 1256.9 | Improved |
| add SPR loss | streaming DQN encoder · Atari 26 (40M frames) effective rank | effective rank: 518.7 -> 526.7 | No clear effect |
| increase prediction depth K from 1 to 5 (ablation) | QRC(lambda)+SPR · Atari 26 (40M frames, human normalized) | IQM return: 0.46 -> 0.61 | Improved |
| use replay buffer size 5 instead of 1 (ablation) | streaming DQN · Atari 26 (40M frames, human normalized) | IQM return: 0.12 -> 0.13 | No clear effect |
Closest papers by meaning
- Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamics (ICML 2024 · similarity 0.86)
- Bridging State and History Representations: Understanding Self-Predictive RL (ICLR 2024 · similarity 0.85)
- Planning, Fast and Slow: Online Reinforcement Learning with Action-Free Offline Data via Multiscale Planners (ICML 2024 · similarity 0.85)
- Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
- RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO (NeurIPS 2026 · similarity 0.85)
- Towards Principled Representation Learning from Videos for Reinforcement Learning (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
