Research map / Video / Video generation
UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds UltraVideo, a 42K-clip (plus 17K long) native 4K/8K text-to-video dataset with ten structured captions per clip, and fine-tunes Wan-T2V-1.3B via LoRA into UltraWan-1K/4K.
Problem
Existing public text-to-video datasets are mostly 720p or lower with short captions, which limits training of native ultra-high-definition (4K/8K) video generation models; generic Wan-T2V extrapolated to high resolution degrades sharply.
Approach
A four-stage curation pipeline (YouTube 4K/8K source collection, statistical filtering of text/black border/exposure/graying, model-based purification with VTSS, motion, CLIP consistency and MLLM attribute checks, and Qwen2.5-VL-72B structured captioning with a Qwen3-4B summary caption), plus a random caption sampling strategy and LoRA fine-tuning of Wan-T2V-1.3B for 1K and 4K generation.
Key results
- UltraWan-1K (LoRA) reaches 67.74% imaging quality and 82.29% object class vs. 55.12% and 33.33% for Wan-T2V-1.3B-1K extrapolation.
- In a human study with 10 subjects, UltraWan-1K was preferred for video quality aesthetics 81.10% vs. 18.90% for official Wan.
- UltraVideo has a 2.3% bad-video failure rate versus 41.5% for Koala-36M on 1000 sampled videos judged by five annotators.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Wan-T2V-1.3B on UltraVideo with LoRA at 1K rather than extrapolating the base model to 1K | Wan-T2V-1.3B · VBench | Imaging Quality: 55.12% -> 67.74% | Helped |
| fine-tune Wan-T2V-1.3B on UltraVideo with LoRA at 1K rather than extrapolating the base model to 1K | Wan-T2V-1.3B · VBench | Overall Consistency: 20.65% -> 23.99% | Helped |
| generate at 4K with LoRA fine-tuning rather than 1K with LoRA fine-tuning | UltraWan · VBench | Imaging Quality: 67.74% -> 71.61% | Helped |
| generate at 4K with LoRA fine-tuning rather than 1K with LoRA fine-tuning | UltraWan · VBench | Overall Consistency: 23.99% -> 22.88% | Hurt |
| use full-parameter fine-tuning instead of LoRA fine-tuning at 1K (ablation) | UltraWan-1K · VBench | Imaging Quality: 67.74% -> 67.39% | No clear effect |
| fine-tune on UltraVideo for native 1K generation rather than use official Wan at 480p | Wan-T2V-1.3B · 1K video generation human preference | Video quality aesthetics preference: 18.90% -> 81.10% | Helped |
| apply the UltraVideo curation process instead of using Koala-36M videos | video dataset · Koala-36M vs UltraVideo sampled videos | Bad video failure rate: 41.5% -> 2.3% | Helped |
Surprising findings
- Scaling the official Wan-T2V-1.3B directly to 1K resolution significantly degrades performance, and some metrics such as Background Consistency and Dynamic Degree are argued to conflict with human perception at high resolution.
Limitations
- UltraWan-4K is trained with only 29 frames and one epoch due to memory and compute limits, and shows more subject artifacts than UltraWan-1K.
- Evaluation is limited to a small-scale Wan-T2V-1.3B model and a VBench subset of about 96 prompts.
Experimental setup
Datasets: UltraVideo Koala-36M
Benchmarks: VBench
Metrics: VBench dimension scores human preference (video quality aesthetics, temporal stabili non-compliance failure rate
Baselines: Wan-T2V-1.3B-480p Wan-T2V-1.3B-1K UltraWan-1K (Full)
Builds on: Wan Koala-36M MiraData Qwen2.5-VL
Closest papers by meaning
- OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (ICLR 2025 · similarity 0.87)
- Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content (CVPR 2025 · similarity 0.87)
- LVD-2M: A Long-take Video Dataset with Temporally Dense Captions (NeurIPS 2024 · similarity 0.87)
- UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset (NeurIPS 2025 · similarity 0.87)
- Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (CVPR 2024 · similarity 0.86)
- ShareGPT4Video: Improving Video Understanding and Generation with Better Captions (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.