Research map / Reinforcement learning / Policy gradient and actor-critic
Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VDFP in place of the conventional DDPG critic | DDPG · HalfCheetah-v1, accumulated reward delays d=16, 32, 64, 128 | max average episode reward: 2675 -> 4823; 1546 -> 4677; 1176 -> 3667; 909 -> 2479 | Helped |
| Use VDFP in place of the conventional DDPG critic | DDPG · Walker2d-v1, accumulated reward delays d=16, 32, 64, 128 | max average episode reward: 653 -> 1822; 583 -> 1824; 404 -> 1628; 325 -> 570 | Helped |
| Replace LSTM trajectory representation with CNN (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5197.03 ± 156.52 -> 5818.60 ± 336.25 | Helped |
| Replace the conditional VAE dynamics model with an MLP (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 2029.00 ± 486.11 | Hurt |
| Replace element-wise product with concatenation in conditional encoding (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 4541.71 ± 104.22 | Hurt |
| Replace the linear trajectory return with convex Leaky-ReLU, α=0.2 (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 4781.18 ± 440.50 | Hurt |
| Replace the linear trajectory return with convex Leaky-ReLU, α=0.5 (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 4985.66 ± 348.08 | Hurt |
| Replace the linear trajectory return with ICNN (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 5646.12 ± 328.61 | No clear effect |
| Replace the linear trajectory return with NE-ICNN (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 5310.23 ± 198.35 | No clear effect |
| Reduce the VAE KL weight from β=1000 to β=100 (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 4794.96 ± 370.02 | Hurt |
| Reduce the VAE KL weight from β=1000 to β=10 (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 3933.33 ± 361.82 | Hurt |
| Remove clipping from generative noise by setting c=∞ (ablation) | VDFP · HalfCheetah-v1 | max average episode reward: 5818.60 ± 336.25 -> 4752.84 ± 328.75 | Hurt |
Closest papers by meaning
- Value-driven Hindsight Modelling (NeurIPS 2020 · similarity 0.91)
- Pretrain Value, Not Reward: Decoupled Value Policy Optimization (ICLR 2026 · similarity 0.91)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- Understanding End-to-End Model-Based Reinforcement Learning Methods as Implicit Parameterization (NeurIPS 2021 · similarity 0.90)
- Critic-Guided Decision Transformer for Offline Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Horizon Adaptive Offline Policy Learning via Value Stitching (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.