Research map / Video / Video-language understanding
DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics
Silin Gao, Hao Zhao, Zeming Chen, Sepideh Mamooler, Antara Raaghavi Bhattacharya, Qiyu Wu, Hiromi Wakaki, Yuki Mitsufuji, Li Mi, Syrielle Montariol, Antoine Bosselut
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| continue pre-training BAGEL on hierarchical state-transition data (DynaVieW) before zero-shot narrative generation | BAGEL-7B-MoT · VinaBench VWP zero-shot | average score: 0.567 -> 0.630 | Improved |
| continue pre-training on state-transition data then SFT on VinaBench | BAGEL-7B-MoT · VinaBench VWP SFT | average score: 0.547 -> 0.610 | Improved |
| add SFT on task data to DynaVieW | DynaVieW · VinaBench VWP | cross-scene consistency (character): 0.550 -> 0.483 | Mixed |
| add multimodal selective attention (ablation) | DynaVieW · VinaBench VWP zero-shot | average score: -0.013 when removed | Improved |
| add schema token re-weighted CE loss (ablation) | DynaVieW · VinaBench VWP zero-shot | average score: -0.004 when removed | Improved |
| use JSON-schema transitions instead of template natural language (ablation) | DynaVieW training data · VinaBench VWP zero-shot | average score: 0.028 average decrease when removed | Improved |
| use fine-grained low-level transitions instead of coarse schema (ablation) | DynaVieW training data · VinaBench VWP zero-shot | average score: 0.064 average decrease when removed | Improved |
| use DynaVieW instead of BAGEL with LLM-generated JSON schema prompts | next-scene control · VinaBench controllability | average score: 0.283 -> 0.604 | Improved |
| convert JSON schema into natural language for BAGEL (ablation) | BAGEL · VinaBench controllability | average score: 0.283 -> 0.275 | No clear effect |
| use DynaVieW instead of BAGEL for world simulation, no task SFT | BAGEL-7B-MoT · LEGO Ego4D | FID: 26.94 -> 26.15 | Improved |
| use DynaVieW instead of BAGEL for world simulation after SFT | BAGEL-7B-MoT · LEGO Epic-Kitchens (SFT) | FID: 11.66 -> 10.31 | Improved |
| use DynaVieW instead of BAGEL (zero-shot) | BAGEL-7B-MoT · LEGO Epic-Kitchens | LPIPS: 41.45 -> 41.78 | Mixed |
Closest papers by meaning
- DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time (NeurIPS 2026 · similarity 0.90)
- DynaMind: Reasoning over Abstract Video Dynamics for Embodied Decision-Making (ICML 2025 · similarity 0.88)
- DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modeling (NeurIPS 2025 · similarity 0.87)
- DyST: Towards Dynamic Neural Scene Representations on Real-World Videos (ICLR 2024 · similarity 0.87)
- Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs (CVPR 2024 · similarity 0.87)
- LogiStory: A Logic-Aware Framework for Multi-Image Story Visualization (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
