Research map / Video / Video generation
SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPIRAL, a closed-loop agentic harness (PlanAgent, VideoGenerator, CriticAgent, context memory) for long-horizon action-conditioned video generation, plus a GRPO-based self-evolving step, and releases the ActVideoGen dataset and benchmark.
Problem
Single-shot, open-loop text/image-to-video models fail on long-horizon action-conditioned generation, producing incomplete actions, hallucinated motions, and temporal drift.
Approach
Wraps a video generator in a closed think-act-reflect loop: a VLM planner decomposes goals into pre/post-condition action steps, a VLM critic scores each segment and triggers local refinement or global replanning, a context memory preserves history, and GRPO fine-tunes the generator using critic rewards with curriculum over horizons.
Key results
- Closed-loop SPIRAL improves overall action quality by 22.6% over the open-loop baseline, with GRPO-based evolving adding a further 4.4%.
- PlanAgent reaches 58.72% accuracy on EgoPlan-Bench, outperforming GRPO-CARE by 3.94%.
- Action completeness rises from 0.347 (baseline) to 0.781 with the full SPIRAL framework in the ablation table.
Surprising findings
- Generic VBench visual and dynamic metrics fluctuate little and can reward open-loop baselines that collapse into short, incomplete motions.
- PlanAgent generalizes worse than GRPO-CARE on out-of-domain EPIC-KITCHENS-to-Ego4D splits, which the authors attribute to DPO fitting the training distribution.
Limitations
- Relies on proprietary VLM/LLM teachers (GPT-5.1, Gemini-3-Pro, GPT-5) for annotation, distillation and evaluation, creating cost and reproducibility concerns.
- Generic visual and dynamic metrics show minimal gains and are judged to be biased toward short-term fidelity.
- Human verification covers only a random subset, reporting 93% annotation agreement.
Experimental setup
Datasets: ActVideoGen-Dataset Ego4D Ego-Exo4D COIN EPIC-KITCHENS GAIA VideoVerse
Benchmarks: ActVideoGen-Bench EgoPlan-Bench VideoGen-RewardBench VBench
Metrics: action completeness action smoothness object interaction physical fidelity task success rate preference accuracy VBench static and dynamic quality
Baselines: Baseline open-loop TI2V generator GRPO-CARE LongLive CogVideoX SkyReels
Builds on: GRPO DPO Stable-Video-Infinity VBench
Closest papers by meaning
- EvoGround: Self-Evolving Video Agents for Video Temporal Grounding (NeurIPS 2026 · similarity 0.86)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.85)
- FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model (ICLR 2025 · similarity 0.85)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.85)
- VisionCreator-R1: A Reflection-Enhanced Native Visual-Generation Agentic Model (NeurIPS 2026 · similarity 0.84)
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.