Research map / Video / Video generation
OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OpenS2V-Nexus, a fine-grained subject-to-video benchmark (OpenS2V-Eval, 180 prompts, three new metrics NexusScore/NaturalScore/GmeScore) and a 5M-clip S2V training dataset (OpenS2V-5M) with GPT-Image-1-synthesized multi-view "Nexus Data".
Problem
Existing subject-to-video (S2V) benchmarks are adapted from text-to-video evaluation and coarse-grained, missing subject naturalness and identity fidelity, while open large-scale S2V training datasets are lacking.
Approach
Evaluate S2V with a seven-category benchmark and metrics that detect and crop the subject in frames (NexusScore), score naturalness with GPT-4o over sampled frames (NaturalScore), and score text relevance with a Qwen2-VL-based embedding model (GmeScore); build a dataset whose reference images come from cross-video associations and GPT-Image-1 multi-view synthesis rather than from the ground-truth
Key results
- Kling1.6 achieves the highest open-domain total score among closed-source models at 54.46% and highest NexusScore (45.92%) and NaturalScore (79.06%) among closed-source models.
- Hailuo-S2V-01 gets the top human-domain Total Score of 60.20%, while the best open-source human-domain model EchoVideo scores 54.52%.
- Validation fine-tune on 300k OpenS2V-5M samples yields a human-domain total score of 52.97% (with Nexus Data) versus 51.67% without, per the paper's ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase VACE model size from 1.3B to 14B | VACE · Open-domain S2V | Total Score: 45.53% -> 52.87% | Helped |
| Increase VACE model size from 1.3B to 14B | VACE · Human-domain S2V | Total Score: 49.02% -> 58.57% | Helped |
| Increase VACE model size from 1.3B to 14B | VACE · Single-domain S2V | Total Score: 47.33% -> 58.00% | Helped |
| Add Nexus Data to OpenS2V-5M training data (ablation) | Wan2.1 1.3B fine-tuned with ConsisID · Human-domain S2V | Total Score: 52.97% -> 51.67% | Hurt |
| Add Nexus Data to OpenS2V-5M training data (ablation) | Wan2.1 1.3B fine-tuned with ConsisID · Human-domain S2V | NaturalScore: 66.80% -> 65.42% | Hurt |
Surprising findings
- VACE-14B has a higher open-domain total score than Phantom-14B on several dimensions, while Phantom-14B shows higher motion; SkyReels-A2 has high NexusScore but low NaturalScore, indicating copy-paste behavior.
- Open-source human-domain models such as ConsisID and Concat-ID have relatively strong FaceSim but suffer significant copy-paste issues on NaturalScore.
Limitations
- Benchmark is small (180 prompts), and closed-source models were evaluated through manual interfaces with one video per sample.
- Validation fine-tune used only 300k samples and a single human identity focus, so the model is not fully trained.
- NaturalScore depends on GPT-4o as a judge, which may introduce bias and cost.
Experimental setup
Datasets: OpenS2V-5M Open-Sora Plan raw videos ConsisID A2 Bench DreamBench
Benchmarks: OpenS2V-Eval
Metrics: NexusScore NaturalScore GmeScore Total Score Aesthetics Motion FaceSim human preference agreement
Baselines: Vidu2.0 Pika2.1 Kling1.6 VACE-1.3B VACE-14B Phantom-1.3B Phantom-14B SkyReels-A2 HunyuanCustom Hailuo-S2V-01 ConsisID Concat-ID
Builds on: VBench ConsisID Grounding DINO SAM2.1 Wan2.1
Closest papers by meaning
- OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (ICLR 2025 · similarity 0.90)
- LVD-2M: A Long-take Video Dataset with Temporally Dense Captions (NeurIPS 2024 · similarity 0.87)
- VBench: Comprehensive Benchmark Suite for Video Generative Models (CVPR 2024 · similarity 0.87)
- Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability, Reproducibility, and Practicality (NeurIPS 2024 · similarity 0.87)
- Video-Bench: Human-Aligned Video Generation Benchmark (CVPR 2025 · similarity 0.87)
- EvalCrafter: Benchmarking and Evaluating Large Video Generation Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.