Research map / Reinforcement learning / MDPs and planning
MICo: Improved representations via sampling-based state similarity for Markov decision processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the MICo representation-shaping loss to Dopamine agents (DQN, Rainbow, QR-DQN, IQN, M-IQN) | DQN, Rainbow, QR-DQN, IQN, M-IQN · Atari 2600 (60 games, 5 seeds) | interquantile mean human-normalized score | Helped |
| add MICo loss to SAC with pixel observations | SAC · DM-Control from pixels (12 tasks) | IQM score | Helped |
| replace DBC bisimulation loss (needs dynamics model) with MICo loss (ablation) | DBC and DBC-Det · DM-Control from pixels | IQM score | Helped |
| minimize MICo loss with MSE instead of Huber loss (ablation) | MICo loss · Atari 2600 | agent performance | Hurt |
| reduced MICo distance as upper bound on value differences vs U^pi and pi-bisimulation | reduced MICo distance · Garnet MDPs | average value bound gap | Helped |
| independent coupling (MICo features) instead of pi-bisimulation optimal coupling | state features for value regression · four-rooms GridWorld | value regression error | No clear effect |
Closest papers by meaning
- Learning Markov State Abstractions for Deep Reinforcement Learning (NeurIPS 2021 · similarity 0.91)
- Bridging State and History Representations: Understanding Self-Predictive RL (ICLR 2024 · similarity 0.88)
- A Generalized Bisimulation Metric of State Similarity between Markov Decision Processes: From Theoretical Propositions to Applications (NeurIPS 2025 · similarity 0.87)
- Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (NeurIPS 2024 · similarity 0.87)
- Towards Principled Representation Learning from Videos for Reinforcement Learning (ICLR 2024 · similarity 0.86)
- Diffusion Spectral Representation for Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.