Research map / Video / Video-language understanding
Generation-for-Understanding with Structured Action Scripts for Embodied Multimodal Learning
Hange Zhou, Mingzhe Zhang, Hongyu Ge, Xiaotang Wang, Enjun Du, Runqing Xu, Yongqi Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes generation-for-understanding with structured action scripts to help VLMs recover fine-grained intermediate actions and result states in embodied videos.
Problem
VLMs used as perception and reasoning modules for embodied intelligence are brittle at understanding fine-grained physical processes in videos, often recognizing the overall task while missing intermediate actions and result states.
Approach
Uses generation of structured action scripts as an auxiliary objective so that the model learns to produce the intermediate actions and result states that govern how an embodied interaction unfolds, improving understanding.
Closest papers by meaning
- Rethinking Video Generation Model for the Embodied World (ICML 2026 · similarity 0.90)
- EC2: Emergent Communication for Embodied Control (CVPR 2023 · similarity 0.89)
- DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control (NeurIPS 2026 · similarity 0.89)
- WorldSimBench: Towards Video Generation Models as World Simulators (ICML 2025 · similarity 0.89)
- 3D-VLA: A 3D Vision-Language-Action Generative World Model (ICML 2024 · similarity 0.88)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
