Research map / Reinforcement learning / Visual and representation RL
ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning in Instructional Videos
Luigi Seminara, Davide Moltisanti, Antonino Furnari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add differentiable Viterbi layer during training (ablation) | ViterbiPlanNet base model · CrossTask (T=3) | Success Rate (%): 32.47 -> 38.09 (VD inference) | Improved |
| add Viterbi decoding or DVL only at inference to a model trained without DVL (ablation) | base model · CrossTask (T=3) | Success Rate (%): 32.47 -> 32.99 / 32.09 / 30.77 | No clear effect |
| use emissions directly (argmax) at inference after DVL training (ablation) | ViterbiPlanNet · CrossTask (T=3) | Success Rate (%): 38.09 -> 20.05 | Hurt |
| add standard Viterbi decoding on top of DVL soft plan (ablation) | ViterbiPlanNet · CrossTask (T=3) | Success Rate (%): 37.66 -> 38.45 | No clear effect |
| enable PKG via guided training rather than conditioning or post-processing (ablation) | ViterbiPlanNet vs KEPP/PlanLLM/SCHEMA · CrossTask (T=3) | SR gain from PKG (%): +5.98 vs +2.56, +1.97, +3.31 | Improved |
| add PKG into LLM prompt (ablation) | Qwen3-30B · CrossTask (T=3) | Success Rate (%): 23.37 -> 23.31 | No clear effect |
| train with DVL and PKG instead of transformer planner (SCHEMA) at small training fractions (ablation) | ViterbiPlanNet · CrossTask (T=3, training data fraction) | Success Rate (%) | Improved |
| ViterbiPlanNet vs SCHEMA when trained on T=6, tested on T=3 | ViterbiPlanNet · CrossTask cross-horizon | Success Rate (%): 16.12 -> 27.77 | Improved |
| ViterbiPlanNet vs MTID under MTID protocol | ViterbiPlanNet · CrossTask (T=3) | Success Rate (%): 40.45 -> 39.75 | No clear effect |
| ViterbiPlanNet vs best baseline on mIoU | ViterbiPlanNet · CrossTask (T=3) | mIoU (%): -0.05 ± 0.27 (CrossTask mIoU) | No clear effect |
Closest papers by meaning
- Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videos (CVPR 2024 · similarity 0.88)
- VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning (ICLR 2025 · similarity 0.88)
- Procedure Knowledge Decoupled Distillation Strategy for Procedure Planning in Instructional Videos (AAAI 2025 · similarity 0.87)
- VG-TVP: Multimodal Procedural Planning via Visually Grounded Text-Video Prompting (AAAI 2025 · similarity 0.87)
- PlanLLM: Video Procedure Planning with Refinable Large Language Models (AAAI 2025 · similarity 0.87)
- RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
