Research map / Video / Video generation
Less is More: Data-Efficient Adaptation for Controllable Text-to-Video Generation
Shihan Cheng, Nilesh Kulkarni, David Hyde, Dmitriy Smirnov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on simple synthetic scene instead of photorealistic scene (ablation) | Wan2.1 fine-tuning (shutter speed) · One-shot data comparison | X-CLIP score: 23.278 -> 24.777 | Improved |
| train on simple synthetic scene instead of photorealistic scene (ablation) | Wan2.1 fine-tuning (aperture) · One-shot data comparison | X-CLIP score: 19.824 -> 25.105 | Improved |
| train on simple synthetic scene instead of photorealistic scene (ablation) | Wan2.1 fine-tuning (color temperature) · One-shot data comparison | X-CLIP score: 24.456 -> 25.015 | Improved |
| train on simple synthetic scene instead of photorealistic scene (ablation) | Wan2.1 fine-tuning (aperture) · One-shot data comparison | VQA score: 0.021 -> 0.343 | Improved |
| train on simple synthetic scene instead of photorealistic scene (ablation) | Wan2.1 fine-tuning (shutter speed) · One-shot data comparison | VQA score: 0.096 -> 0.352 | Improved |
| train on simple synthetic scene instead of photorealistic scene (ablation) | Wan2.1 fine-tuning · One-shot data comparison, FEP drift | SSF / SS-FD drift rate | Improved |
| discard backbone LoRA from shallow two-thirds of blocks at inference (decoupled) (ablation) | Wan2.1 + adapter (aperture) · SVP 96-prompt suite | VQA score: 0.427 -> 0.513 | Improved |
| decoupled instead of full-LoRA inference (ablation) | Wan2.1 + adapter (shutter speed) · SVP 96-prompt suite | VQA score: 0.453 -> 0.521 | Improved |
| decoupled instead of full-LoRA inference (ablation) | Wan2.1 + adapter (color temperature) · SVP 96-prompt suite | VQA score: 0.550 -> 0.532 | Hurt |
| decoupled instead of full-LoRA inference (ablation) | Wan2.1 + adapter (shutter speed) · SVP 96-prompt suite | Dynamic degree: 0.688 -> 0.469 | Improved |
| decoupled instead of full-LoRA inference (ablation) | Wan2.1 + adapter · SVP 96-prompt suite | X-CLIP score: 25.295 -> 25.587 (shutter) | Mixed |
| joint training of backbone LoRA with conditional adapter instead of adapter-only (ablation) | Wan2.1 adapter · Block 27 singular value spectrum | effective rank of y_cond: high-rank -> effective rank of 1 | Improved |
Closest papers by meaning
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.89)
- ControlVideo: Training-free Controllable Text-to-video Generation (ICLR 2024 · similarity 0.89)
- Encapsulated Composition of Text-to-Image and Text-to-Video Models for High-Quality Video Synthesis (CVPR 2025 · similarity 0.88)
- ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Models (CVPR 2025 · similarity 0.88)
- TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models (CVPR 2024 · similarity 0.88)
- Training-free Camera Control for Video Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
