Research map / Video / Video generation
Rewind and Render: Towards Factually Accurate Text-to-Video Generation with Distilled Knowledge Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FACT-V, a demo system that adds LLM and web-search knowledge retrieval and prompt augmentation to text-to-video pipelines to improve factual accuracy and interpretability.
Problem
Text-to-video models struggle with factual accuracy for knowledge-dense, entity-specific, or time-sensitive prompts, and retraining with up-to-date data is costly.
Approach
Chain LLM-based knowledge distillation guided by a manually curated entity-type ontology, retrieve facts from GPT-4o and Perplexity, filter facts with visual implications, and rewrite the prompt before passing it to a T2V model, while showing the retrieved knowledge to users.
Key results
- Preliminary human preference evaluation shows a notable improvement in factuality with FACT-V over the standard T2V pipeline.
- Alignment with the prompt slightly exceeds parity with FACT-V, with high annotator agreement.
- Evaluation is small in scale and focused on challenging prompts; no specific numeric results are reported in the text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dynamically retrieved, distilled knowledge and prompt augmentation | FACT-V with a T2V model · challenging prompts | factuality | Helped |
| add dynamically retrieved, distilled knowledge and prompt augmentation | FACT-V with a T2V model · challenging prompts | alignment | Helped |
Limitations
- Evaluation is small-scale with a preliminary preference study and no reported numeric metrics.
- Relies on closed-source API LLMs and web search for retrieval.
- Specific T2V models used for generation are not named in the text.
Experimental setup
Datasets: web search results via Perplexity API small set of challenging prompts
Benchmarks: small-scale preferential human evaluation of challenging prompts
Metrics: factuality preference alignment preference annotator agreement
Baselines: T2V model without FACT-V augmentation
Builds on: retrieval-augmented generation Khot et al. 2023 decomposed prompting
Closest papers by meaning
- Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding (NeurIPS 2025 · similarity 0.87)
- T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation (CVPR 2025 · similarity 0.86)
- Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrieval (AAAI 2025 · similarity 0.86)
- The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation (CVPR 2025 · similarity 0.86)
- Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval (CVPR 2025 · similarity 0.86)
- PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.