Research map / Reinforcement learning / Policy gradient and actor-critic
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Angelos Filos, Eszter Vértes, Zita Marinho, Gregory Farquhar, Diana Borsa, Abram Friesen, Feryal Behbahani, Tom Schaul, Andre Barreto, Simon Osindero
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the IVE disagreement bonus to Dreamer planning | Dreamer · Walker Walk, η=0.3 | episodic return: 570±60 -> 1000±00 | Improved |
| Add the IVE disagreement bonus to Dreamer planning | Dreamer · Walker Walk, η=0.5 | episodic return: 80±50 -> 330±70 | Improved |
| Use the IVE mean rather than the base policy for planning (ablation) | Dreamer · Walker Walk, η=0.2 | episodic return: 720±10 -> 860±40 | Improved |
| Replace the IVE mean with the 1-MPV (ablation) | VPN(5) · Asterix | episodic return: 18.3±0.2 -> 15.1±0.6 | Hurt |
| Replace the IVE mean with the 5-MPV (ablation) | VPN(5) · Asterix | episodic return: 18.3±0.2 -> 7.1±2.3 | Hurt |
| Replace the IVE mean with the 1-MPV (ablation) | VPN(5) · Breakout | episodic return: 22.0±0.7 -> 13.8±0.8 | Hurt |
| Replace the IVE mean with the 5-MPV (ablation) | VPN(5) · Breakout | episodic return: 22.0±0.7 -> 4.2±2.3 | Hurt |
| Replace the IVE mean with the 1-MPV (ablation) | VPN(5) · Freeway | episodic return: 49.4±0.5 -> 49.1±0.7 | No clear effect |
| Replace the IVE mean with the 5-MPV (ablation) | VPN(5) · Freeway | episodic return: 49.4±0.5 -> 24.3±4.2 | Hurt |
| Replace the IVE mean with the 1-MPV (ablation) | VPN(5) · Seaquest | episodic return: 8.6±0.3 -> 4.7±0.9 | Hurt |
| Replace the IVE mean with the 5-MPV (ablation) | VPN(5) · Seaquest | episodic return: 8.6±0.3 -> 1.2±1.4 | Hurt |
| Use self-inconsistency to seek novel states | Tabular gridworld policy · 5×5 gridworld | probability of reaching the novel state | Improved |
Closest papers by meaning
- Universal Value-Function Uncertainties (ICLR 2026 · similarity 0.88)
- Exploration via Epistemic Value Estimation (AAAI 2023 · similarity 0.88)
- Self-Consistent Models and Values (NeurIPS 2021 · similarity 0.88)
- Calibrated Value-Aware Model Learning with Probabilistic Environment Models (ICML 2025 · similarity 0.86)
- The Value Equivalence Principle for Model-Based Reinforcement Learning (NeurIPS 2020 · similarity 0.85)
- When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
