Research map / Reinforcement learning / Visual and representation RL
Stackelberg Coupling of Online Representation Learning and Reinforcement Learning
Fernando Martinez, Tao Li, Yingdong Lu, Juntao Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Stackelberg two-timescale coupling (SCORER) on DQN | DQN · MinAtar Breakout | final IQM return: 19.16 -> 65.69 | Improved |
| Use SCORER on DuelingDQN | DuelingDQN · MinAtar Asterix | final IQM return: 39.22 -> 52.28 | Improved |
| Use SCORER on DQN | DQN · MinAtar Asterix | final IQM return: 54.95 -> 54.78 | No clear effect |
| Use SCORER on PQN | PQN · MinAtar (4 games) | final IQM return: Breakout 69.97 -> 71.16; Freeway 61.77 -> 61.05 | Mixed |
| Invert roles so perception leads and control follows (ablation) | SCORER · MinAtar | aggregate IQM | Hurt |
| Use Bellman error variance as follower objective instead of MSBE (ablation) | SCORER · MinAtar | aggregate IQM | Improved |
| Add hierarchical timescale separation versus synchronous coupling of two networks (ablation) | SCORER · MinAtar | aggregate IQM | Improved |
| Use faster encoder learning rate in monolithic DQN (per-layer LR) (ablation) | DQN · MinAtar | aggregate IQM | Improved |
| Use SCORER on R2D2 | R2D2 · MiniGrid Four Rooms | success rate: 0% -> 97% success | Improved |
| Use SCORER with linear networks | Linear TD · Baird's counterexample | loss stability | Improved |
| Use SCORER with Bootstrapped DQN | Bootstrapped DQN · Stochastic Deep Sea (depth 10) | solved fraction: ~20% -> near-complete | Improved |
Closest papers by meaning
- The Value-Improvement Path: Towards Better Representations for Reinforcement Learning (AAAI 2021 · similarity 0.88)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.87)
- Harnessing Structures for Value-Based Planning and Reinforcement Learning (ICLR 2020 · similarity 0.87)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.87)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.86)
- Reinforcement Learning from Imperfect Corrective Actions and Proxy Rewards (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
