Research map / Architectures and efficiency / RNNs and normalization
Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DT-VIN, a Value Iteration Network variant with observation- and state-dependent transition kernels and an adaptive highway loss, enabling training of 5000-layer planners for long-horizon maze navigation, continuous control and rover navigation.
Problem
VINs fail to scale to large-scale, long-horizon planning (e.g., 100x100 mazes requiring thousands of planning steps) due to limited latent MDP representational capacity and shallow planning depth with vanishing gradients.
Approach
Replace VIN's invariant latent transition kernel with a fully dynamic kernel generated by a CNN from the observation and latent state, apply softmax normalization to kernels, and add an adaptive highway loss that attaches losses to layers n >= l (expert path length) so very deep planning modules train stably.
Key results
- On 100x100 mazes with shortest path lengths 1200-1800, DT-VIN (5000 layers) reaches 88.65% success rate, while VIN and GPPN reach 0% and Highway VIN 0%.
- On 35x35 mazes with shortest path length 200-300, DT-VIN achieves 99.77% success rate versus 54.40% for Highway VIN and 0% for VIN and GPPN.
- On Ant Maze 100x100 (continuous control), DT-VIN achieves 93.26% success rate versus 61.78% for the best baseline GPPN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the fully dynamic transition kernel instead of an invariant kernel (ablation) | DT-VIN · 35×35 maze navigation across shortest-path lengths and obstacle counts | success rate | Helped |
| Add observation-dependent transition dynamics (ablation) | DT-VIN latent transition kernel · 35×35 maze navigation | success rate | Helped |
| Add latent-state-dependent transition dynamics (ablation) | DT-VIN latent transition kernel · 35×35 maze navigation | success rate | Helped |
| Add softmax normalization to the latent transition kernel (ablation) | DT-VIN · 35×35 maze navigation | success rate | Helped |
| Add adaptive highway loss (ablation) | DT-VIN · 35×35 maze navigation | success rate | Helped |
| Apply adaptive highway loss rather than highway loss on every hidden layer (ablation) | DT-VIN · 35×35 maze navigation | success rate | Helped |
| Apply adaptive highway loss for layers n≥l rather than only at n=l (ablation) | DT-VIN · 35×35 maze navigation | success rate | Helped |
| Use final-loss skip connections rather than intermediate-layer skip connections (ablation) | DT-VIN · 35×35 maze navigation | success rate | Helped |
| Increase planning depth from 100 to 300 (ablation) | DT-VIN · 35×35 maze, SPL 200 | success rate | Helped |
| Increase planning depth from 300 to 600 (ablation) | DT-VIN · 35×35 maze, SPL 300 | success rate | Helped |
| Set highway-loss interval J from 10 to 1 (ablation) | DT-VIN · 35×35 maze, SPL [200,300] | success rate: 99.77_{\pm 0.23} -> 99.78_{\pm 0.21} | No clear effect |
| Set highway-loss interval J from 10 to 50 (ablation) | DT-VIN · 35×35 maze, SPL [200,300] | success rate: 99.77_{\pm 0.23} -> 99.69_{\pm 0.27} | No clear effect |
Surprising findings
- Removing the softmax normalization on the latent transition kernel causes the model to fail on all tasks due to exploding gradients and NaN values.
- Using the L1 distance heuristic or non-expert path length in place of the true expert path length for the highway loss causes only a small performance drop (about 3%).
Limitations
- Increased computational cost relative to VIN and Highway VIN due to deep network scale.
- Transition mapping module is a single CNN layer; more sophisticated modules are left to future work.
- Upper bound on scalability with depth is unknown.
Experimental setup
Datasets: GPPN 2D maze dataset (25000 train / 5000 val / 5000 test) D4RL PointMaze-Large D4RL AntMaze-Large-Play Apollo 17 orthomosaic terrain images and elevation data ViZDoom 3D maze
Benchmarks: 2D maze navigation (15, 35, 100) 3D ViZDoom navigation D4RL Point Maze and Ant Maze Apollo 17 rover navigation
Metrics: success rate optimality rate
Baselines: VIN GPPN Highway VIN CNN+A*
Builds on: Value Iteration Networks Highway VIN GPPN Highway Networks
Closest papers by meaning
- Highway Value Iteration Networks (ICML 2024 · similarity 0.91)
- Value Iteration Networks (NeurIPS 2016 · similarity 0.91)
- Gated Path Planning Networks (ICML 2018 · similarity 0.91)
- Scaling up and Stabilizing Differentiable Planning with Implicit Differentiation (ICLR 2023 · similarity 0.89)
- Universal Value Iteration Networks: When Spatially-Invariant Is Not Universal (AAAI 2020 · similarity 0.89)
- HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planning (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.