Research map / Architectures and efficiency / RNNs and normalization
SHIELD: Multi-task Multi-distribution Vehicle Routing Solver with Sparsity and Hierarchy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
placeholder
Approach
Combines Mixture-of-Depths routing in the autoregressive decoder (only a fraction of current-node tokens pass each layer) to enforce compute sparsity with a context-prompted, EM-style soft clustering layer that summarizes unvisited nodes into a few cluster representations, arguing both regularize generalization.
Key results
- SHIELD achieves in-task in-distribution gap of 2.37% on MTMDVRP50 versus 3.15% for MVMoE and 2.73% for MVMoE-Deeper, with best results on all splits.
- On MTMDVRP100 out-task out-distribution, SHIELD reaches 6.27% gap versus 8.51% for MVMoE.
- MVMoE-Deeper runs out of memory on MTMDVRP100, while SHIELD trains on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contextual clustering to SHIELD-MoD (ablation) | SHIELD-MoD · MTMDVRP100 out-task out-dist | Gap: 6.9029% -> 6.2727% | Helped |
| increase MoD capacity from 10% to 20% (ablation) | SHIELD · MTMDVRP50 in-task in-dist | Gap: 2.3747% -> 2.2268% | Helped |
| increase MoD capacity from 10% to 30% (ablation) | SHIELD · MTMDVRP50 in-task out-dist | Gap: 2.7376% -> 2.9608% | Hurt |
| increase MoD capacity from 10% to 40% (ablation) | SHIELD · MTMDVRP50 out-task out-dist | Gap: 6.3520% -> 6.7831% | Hurt |
| increase cluster centers from 5 to 10 (ablation) | SHIELD · MTMDVRP50 out-task out-dist | Gap: 6.3520% -> 7.0129% | Hurt |
| increase cluster centers from 5 to 20 (ablation) | SHIELD · MTMDVRP50 out-task out-dist | Gap: 6.3520% -> 10.8044% | Hurt |
| replace the MoE encoder with six MoD encoder layers (ablation) | SHIELD · MTMDVRP50 out-task out-dist | Gap: 6.3520% -> 12.9905% | Hurt |
| replace dense decoder layers with sparse MoD layers | MVMoE-Deeper · MTMDVRP100 in-task in-dist | Gap: OOM -> 2.5443% | Helped |
Surprising findings
- Replacing encoder layers with MoD layers degrades performance substantially even when doubling layers, so sparsity helps only in the decoder.
- Increasing the number of clusters from 5 to 20 hurts out-task generalization (gap rises to 11.3%).
Limitations
- Evaluated only on MTMDVRP with 6 training tasks and 3 training maps, with gaps relative to HGS/OR-Tools rather than optimal solutions.
- Sparsity level and cluster count are tuned on the same test set; INViT and MoD-encoder variants degrade sharply.
Experimental setup
Datasets: USA13509 JA9847 BM33708 KZ9976 SW24978 EG7146 FI10639 GR9882 VM22775
Benchmarks: MTMDVRP50 MTMDVRP100 MTMDVRP200 CVRPLib Set-X
Metrics: objective (tour length) optimality gap
Baselines: POMO-MTVRP MVMoE MVMoE-Light MVMoE-Deeper SHIELD-MoD INViT HGS OR-Tools
Builds on: MVMoE POMO Mixture-of-Depths INViT
Closest papers by meaning
- MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts (ICML 2024 · similarity 0.89)
- Multi-Task Vehicle Routing Solver via Mixture of Specialized Experts under State-Decomposable MDP (NeurIPS 2025 · similarity 0.88)
- MTL-KD: Multi-Task Learning Via Knowledge Distillation for Generalizable Neural Vehicle Routing Solver (NeurIPS 2025 · similarity 0.87)
- Rethinking Light Decoder-based Solvers for Vehicle Routing Problems (ICLR 2025 · similarity 0.86)
- A Mixed-Curvature based Pre-training Paradigm for Multi-Task Vehicle Routing Solver (ICML 2025 · similarity 0.86)
- SPACE: Unifying Symmetric and Asymmetric Routing Problems for Generalist Neural Solver (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.