Research map / Video / Video generation
MVTokenFlow: High-quality 4D Content Generation using Multiview Token Flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates 4D content from monocular video by running Era3D multiview diffusion per frame, reconstructing a coarse dynamic Gaussian field, then regenerating multiview images with rendered 2D-flow token propagation and refining the field.
Problem
Creating spatially and temporally consistent 4D content from monocular video, where per-frame multiview diffusion lacks temporal consistency across timesteps.
Approach
Extends 2D token flow to multiview diffusion: rendered 2D flows from a coarse dynamic 3D Gaussian field guide token propagation (feature reuse across keyframes) during regeneration of multiview images, which then refine the 4D field; no 4D training data is needed.
Key results
- On Consistent4D view synthesis, MVTokenFlow reports LPIPS 0.04, PSNR 31.27, SSIM 0.97 versus STAG4D's 0.06, 30.79, 0.94.
- Spatial consistency CLIP score 0.91, tied with STAG4D (0.91) and above SC4D (0.90) and Consistent4D (0.89).
- Ablation reports that flow propagation improves temporal consistency (FVD) and normal loss improves multi-view consistency (CLIP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MVTokenFlow instead of STAG4D | 4D content generation on Consistent4D · Consistent4D dataset | LPIPS: 0.06 -> 0.04 | Helped |
| Use MVTokenFlow instead of STAG4D | 4D content generation on Consistent4D · Consistent4D dataset | PSNR: 30.79 -> 31.27 | Helped |
| Use MVTokenFlow instead of STAG4D | 4D content generation on Consistent4D · Consistent4D dataset | SSIM: 0.94 -> 0.97 | Helped |
| Use MVTokenFlow instead of STAG4D | 4D content generation on Consistent4D · Consistent4D dataset | CLIP: 0.91 -> 0.91 | No clear effect |
| Add token propagation with 2D flows (ablation) | MVTokenFlow multiview video generation · Multiview videos | temporal consistency | Helped |
| Add flow loss (ablation) | MVTokenFlow dynamic Gaussian field · Rendered front view | optical flow quality | Helped |
| Add regeneration and refinement (ablation) | MVTokenFlow dynamic Gaussian field · 4D content generation | generation quality | Helped |
| Add normal loss (ablation) | MVTokenFlow dynamic Gaussian field · Consistent4D dataset | multi-view consistency | Helped |
Surprising findings
- CLIP spatial consistency score is tied with STAG4D (0.91), so the method's advantage on that metric is not clear-cut.
Limitations
- Limited by the Era3D multiview diffusion model on complex objects and uncommon viewpoints.
- Token propagation is applied only in early denoising stages because optical flow cannot cover occluded regions.
- Training time (120 min) is comparable to Consistent4D and longer than several baselines.
Experimental setup
Datasets: Consistent4D self-collected internet videos
Benchmarks: Consistent4D
Metrics: LPIPS PSNR SSIM CLIP score FVD
Baselines: Consistent4D SC4D STAG4D
Builds on: Era3D token flow SC-GS 3D Gaussian Splatting RAFT
Closest papers by meaning
- SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency (ICLR 2025 · similarity 0.89)
- Diffusion$^2$: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models (ICLR 2025 · similarity 0.88)
- 4Diffusion: Multi-view Video Diffusion Model for 4D Generation (NeurIPS 2024 · similarity 0.88)
- EG4D: Explicit Generation of 4D Object without Score Distillation (ICLR 2025 · similarity 0.88)
- Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.87)
- Vivid-ZOO: Multi-View Video Generation with Diffusion Model (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.