Research map / Video / Video generation
GeoVideo: Introducing Geometric Regularization into Video Generation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Augments CogVideoX-5B with a jointly generated depth branch and a multi-view geometric consistency loss (lifting depth to a global point cloud and reprojecting) to improve 3D consistency of generated videos.
Problem
Video diffusion models operate in 2D latent space and produce temporally inconsistent geometry, shape deformation, and implausible motion; they lack explicit 3D structure modeling.
Approach
Jointly generate per-frame depth alongside RGB in a shared latent space (with zero-initialized dual input/output projections), and add a reprojection-based cross-frame geometric loss that lifts predicted depths and camera poses into a global point cloud and penalizes depth discrepancies, gated by a motion probability map for dynamic content.
Key results
- On DL3DV multi-view consistency, Ours (T2V) reaches MVCS 77.2 and reprojection error 2.52 vs. 61.2 and 4.58 for CogVideoX-5B.
- Text-to-3D scene comparison on DL3DV: Ours achieves FID 72.78 vs. 79.96 for LucidDreamer, and CLIPScore 33.84 vs. 31.42 for SplatFlow.
- T2V FVD drops from 145.3 (CogVideoX-5B) to 122.7 and I2V FVD from 139.8 to 120.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add geometric regularization loss (ablation) | GeoVideo T2V · DL3DV | MVCS: 71.3 -> 77.2 | Helped |
| add geometric regularization loss (ablation) | GeoVideo T2V · DL3DV | Reprojection Error: 3.36 -> 2.52 | Helped |
| add geometric regularization loss (ablation) | GeoVideo T2V · T2V evaluation videos | FVD: 134.2 -> 122.7 | Helped |
| add geometric regularization loss (ablation) | GeoVideo I2V · I2V evaluation videos | FVD: 128.0 -> 120.5 | Helped |
| add motion probability map (ablation) | GeoVideo T2V · T2V evaluation videos | FVD: 131.6 -> 122.7 | Helped |
| add motion probability map (ablation) | GeoVideo I2V · I2V evaluation videos | FVD: 126.5 -> 120.5 | Helped |
| jointly model depth and appearance (ablation) | GeoVideo · I2V video generation | video quality | Helped |
Limitations
- Evaluation relies on VGGT-based metrics (MVCS, reprojection error) that the method's own supervision signals are related to, so the geometric metrics may be biased toward the method.
- Dynamic-video training data depends on pseudo-labels from MegaSaM and Video Depth Anything rather than ground truth.
- Comparisons with 3D generation baselines use video-derived metrics and perceptual scores without statistical significance reporting.
Experimental setup
Datasets: DL3DV-10K Pexels videos
Benchmarks: DL3DV VBench
Metrics: MVCS reprojection error FID CLIPScore NIQE BRISQUE FVD subject consistency background consistency motion smoothness
Baselines: CogVideoX-5B Director3D LucidDreamer SplatFlow
Builds on: CogVideoX Marigold VGGT ControlNet Video Depth Anything
Closest papers by meaning
- Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach (NeurIPS 2025 · similarity 0.90)
- Diffusion$^2$: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models (ICLR 2025 · similarity 0.89)
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.89)
- Scene Splatter: Momentum 3D Scene Generation from Single Image with Video Diffusion Model (CVPR 2025 · similarity 0.88)
- Diffusion4D: Fast Spatial-temporal Consistent 4D generation via Video Diffusion Models (NeurIPS 2024 · similarity 0.88)
- LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.