Research map / Video / Video generation
Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Prompts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Movie Weaver is a tuning-free video diffusion model for multi-concept video personalization that uses anchored text tokens and concept embeddings to link each reference image to its prompt description, avoiding identity blending.
Problem
Extending single-concept video personalization to multiple reference concepts (face, body, animal) causes identity blending, where composite characters fuse attributes from multiple references because cross attention cannot link each description to its reference image.
Approach
Insert unique anchor tokens ([R1], [R2]) after each concept description in the prompt to link it to an ordered reference image, and add a uniform learned concept embedding to each reference image's tokens to encode image order, without architectural changes beyond these inputs.
Key results
- Single-face personalization: Movie Weaver reaches DINO-I 0.421 and CLIP-I 0.659 vs ID-Animator 0.405 and 0.642, and temporal consistency 0.997 vs 0.985.
- Concept embeddings raise the human-study separable-identity rate from 42.9% (baseline) to 98.2%; adding anchored prompts reaches 99.3%.
- Movie Weaver shows better identity preservation than Vidu 1.5 on face-body-animal and two-face-body examples, where Vidu 1.5 exhibits identity blending (qualitative, human-selected examples).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Movie Weaver instead of ID-Animator | Single-face video personalization · Single-face personalization (97 examples) | CLIP-T: 0.274 -> 0.293 | Helped |
| use Movie Weaver instead of ID-Animator | Single-face video personalization · Single-face personalization (97 examples) | CLIP-I: 0.642 -> 0.659 | Helped |
| use Movie Weaver instead of ID-Animator | Single-face video personalization · Single-face personalization (97 examples) | DINO-I: 0.405 -> 0.421 | Helped |
| use Movie Weaver instead of ID-Animator | Single-face video personalization · Single-face personalization (97 examples) | Temporal Consistency: 0.985 -> 0.997 | Helped |
| add concept embeddings to the baseline without anchored prompts (ablation) | Movie Weaver · Two-face personalization (300 image pairs) | sep yes: 42.9% -> 98.2% | Helped |
| add anchored prompts to concept embeddings (ablation) | Movie Weaver · Two-face personalization (300 image pairs) | face1 sim: 58.8 -> 66.8 | Helped |
| add anchored prompts to concept embeddings (ablation) | Movie Weaver · Two-face personalization (300 image pairs) | face2 sim: 41.9 -> 66.1 | Helped |
| replace concept embeddings with per-token positional embeddings (ablation) | Movie Weaver · Two-face personalization | sep yes: 98.2% -> 90.5% | Hurt |
| add two-face data to one-face pretraining (ablation) | Movie Weaver · Single-face personalization human study (300 datapoints) | face sim: 13.1 -> 29.5 | Helped |
| add face-body-animal data to one-face and two-face pretraining (ablation) | Movie Weaver · Single-face personalization human study (300 datapoints) | face sim: 29.5 -> 46 | Helped |
| add two-face data to one-face pretraining (ablation) | Movie Weaver · Single-face personalization human study (300 datapoints) | face cons: 70 -> 81.3 | Helped |
| add face-body-animal data to one-face and two-face pretraining (ablation) | Movie Weaver · Single-face personalization human study (300 datapoints) | face cons: 81.3 -> 91 | Helped |
Surprising findings
- Positional embedding per token achieved 90.5% separable rate versus 98.2% for the uniform concept embedding, so per-set embedding beat per-token positional encoding.
- Identity blending was less apparent in face-body-animal configurations, since human and animal embeddings are distinct.
Limitations
- Reference images can dominate generation, causing reduced motion, 'big-face' videos, and poor prompt alignment.
- Struggles to generalize to configurations not seen in training, e.g., three people when training had at most two.
- Multi-concept evaluation relies on human studies and qualitative comparisons with limited quantitative baselines.
Experimental setup
Datasets: Shutterstock Video (curated 230K pretraining videos) manually selected finetuning videos
Benchmarks: single-face personalization test set (97 videos) multi-concept personalization test set (5 configurations, 300+ pairs each)
Metrics: CLIP-T CLIP-I DINO-I temporal consistency human study sep yes face similarity face consistency
Baselines: Vidu 1.5 ID-Animator DreamVideo naive multi-reference extension
Builds on: Movie Gen flow matching IP-Adapter
Closest papers by meaning
- Multi-subject Open-set Personalization in Video Generation (CVPR 2025 · similarity 0.87)
- Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models (CVPR 2024 · similarity 0.86)
- VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Models (CVPR 2025 · similarity 0.86)
- ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidance (ICLR 2025 · similarity 0.86)
- CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities (AAAI 2025 · similarity 0.85)
- MaSC: A Masked Similarity Metric for Evaluating Concept-Driven Generation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.