Research map / Video / Video generation
Meta Inverse Prompting for Video Generative Models
Yerin Jung, Jongheon Jeong
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Meta Inverse Prompting, a method for recovering controllable textual conditions from generated videos to invert the usual condition-to-video pipeline.
Problem
Video generative workflows (text-to-video, image-to-video, reference-guided synthesis) are one-directional, offering no mechanism to recover controllable textual conditions from a video.
Approach
Introduces an inverse prompting approach that recovers textual conditions from videos, framed as a meta-level inversion of video generation pipelines; details beyond the abstract are unavailable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add meta-update (meta-learned initialization) to soft-prompt optimization (ablation) | MVIP · MSR-VTT | ViCLIP V-V: 0.8314 -> 0.8434 | Improved |
| use Best-of-N (N=4) candidate selection in inner loop (ablation) | MVIP · MSR-VTT | DINOv2: 0.6174 -> 0.6516 | Improved |
| initialize soft tokens from natural instruction embeddings rather than random (ablation) | MVIP · Held-out validation subset | ViCLIP V-V: 0.9064 -> 0.9193 | Improved |
| increase soft prompt length from 4 to 16 tokens (ablation) | MVIP · Held-out validation subset | ViCLIP V-V: 0.9129 -> 0.9193 | Improved |
| increase soft prompt length from 4 to 8 tokens (ablation) | MVIP · Held-out validation subset | ViCLIP V-V: 0.9129 -> 0.9045 | Hurt |
| increase inner-loop steps K from 1 to 3 during meta-training (ablation) | MVIP · Held-out validation subset | ViCLIP V-V: 0.9193 -> 0.9174 | No clear effect |
| add one test-time adaptation step (K=1) (ablation) | MVIP · Real videos | ViCLIP V-V: 0.8328 -> 0.8392 | Improved |
| add a second test-time adaptation step (K=2) (ablation) | MVIP · Real videos | ViCLIP V-V: 0.8392 -> 0.8322 | Hurt |
| add iterative ARPO-style refinement to captioning | Captioning (Qwen2.5-VL) · Real videos | ViCLIP V-V: 0.7977 -> 0.8047 | Improved |
| use stronger VLM captioner (Qwen3.5-9B instead of Qwen2.5-VL-7B) | Captioning baseline · MSR-VTT | ViCLIP V-V: 0.8021 -> 0.8312 | Improved |
| add a more detailed captioning instruction | Captioning (Qwen3.5-9B) · MSR-VTT | ViCLIP V-V: 0.8312 -> 0.8123 | Hurt |
| transfer the meta-prompt to a different generator (CogVideoX) without retraining | MVIP · Wan2.2 -> CogVideoX | ViCLIP V-V: 0.8222 -> 0.8709 | Improved |
Closest papers by meaning
- PromptCoT: Align Prompt Distribution via Adapted Chain-of-Thought (CVPR 2024 · similarity 0.87)
- VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis (ICLR 2025 · similarity 0.87)
- VideoDirector: Precise Video Editing via Text-to-Video Models (CVPR 2025 · similarity 0.87)
- The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation (CVPR 2025 · similarity 0.87)
- Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.87)
- EditInfinity: Image Editing with Binary-Quantized Generative Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
