Research map / Vision-language and multimodal / Robot action and navigation
"Propose, Assess, Search: Harnessing LLMs for Goal-Oriented Planning in Instructional Videos"
Md Mohaiminul Islam, Tushar Nagarajan, Huiyu Wang, FU-JEN CHU, Kris Kitani, Gedas Bertasius, Xitong Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add propose-assess-search planning on top of LLM prompting (zero-shot, COIN VPA T=3) | Llama-2-70B LLM Baseline · COIN VPA (zero-shot, T=3) | success rate: 2.4 -> 15.3 | Improved |
| add propose-assess-search planning (few-shot, COIN VPA T=3) | Llama-2-70B LLM Baseline · COIN VPA (few-shot, T=3) | success rate: 10.2 -> 21.8 | Improved |
| add assessment and search over the multi-candidate LLM agent baseline | LLM Agent Baseline · COIN VPA (zero-shot, T=3) | success rate: 3.1 -> 15.3 | Improved |
| few-shot VidAssist vs fully supervised VLaMP | VidAssist · COIN VPA T=4 | success rate: 9.0 -> 13.8 | Improved |
| few-shot VidAssist vs fully supervised LFP on COIN PP | VidAssist · COIN PP T=3 | success rate: 30.64 -> 29.20 | Hurt |
| add LLM-based partial plan evaluation value function (ablation) | VidAssist · COIN few-shot, VPA T=3 | success rate: 16.96 -> 21.08 | Improved |
| use partial plan evaluation alone vs generation score alone (ablation) | VidAssist value functions · COIN few-shot, VPA T=3 | success rate: 11.60 -> 16.10 | Improved |
| add few-shot task graph value function (ablation) | VidAssist · COIN few-shot, PP T=4 | success rate: 19.03 -> 20.78 | Improved |
| scale LLM from Llama-2-7B to Llama-2-70B (ablation) | VidAssist · COIN few-shot, VPA T=4 | success rate: 11.71 -> 13.80 | Improved |
| increase in-context examples from 0 to 3 (ablation) | VidAssist · COIN VPA T=3 | success rate: 15.30 -> 21.08 | Improved |
| use ground-truth action history instead of predicted history (ablation) | VidAssist · COIN VPA | success rate: T=3: 21.08 -> 41.33 | Improved |
Closest papers by meaning
- Procedure Knowledge Decoupled Distillation Strategy for Procedure Planning in Instructional Videos (AAAI 2025 · similarity 0.88)
- Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videos (CVPR 2024 · similarity 0.88)
- PlanLLM: Video Procedure Planning with Refinable Large Language Models (AAAI 2025 · similarity 0.88)
- SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos (ICLR 2024 · similarity 0.87)
- Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks (ICML 2025 · similarity 0.86)
- VG-TVP: Multimodal Procedural Planning via Visually Grounded Text-Video Prompting (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
