Research map / Video / Video generation
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free, pose-free method that turns a monocular video from an off-the-shelf video diffusion model (Zeroscope) into a stereoscopic video by depth warping and frame-matrix diffusion inpainting with disocclusion boundary re-injection.
Problem
Generating 3D stereoscopic video from monocular generated videos is under-explored; existing novel-view-synthesis methods depend on unstable camera pose estimation and cannot hallucinate disoccluded content, and stereo training data is scarce.
Approach
Warp the generated left-eye video to multiple evenly spaced camera views to form a frame matrix (rows = views, columns = time), then jointly denoise rows and columns with resampling in a video diffusion model, and re-inject denoised content into disoccluded regions of the latent at each step to avoid dark-region artifacts from VAE encoding.
Key results
- In the human study, Ours scored 5.24 on stereo effect and 5.35 on overall experience, above ProPainter (4.81, 4.66), E2FGVI (4.79, 4.67), RoDynRF, and DynIBaR.
- Ours reached the best CLIP semantic consistency of 96.44, versus 95.29 for ProPainter and 96.03 for RoDynRF; removing the frame matrix dropped it to 95.81.
- Removing disocclusion boundary re-injection dropped the CLIP score to 95.60; the method is reported to fail beyond roughly 20cm baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVG instead of E2FGVI | Stereoscopic video generation · User study | Stereo Effect: 4.79 (1.08) -> 5.24 (0.94) | Helped |
| Use SVG instead of E2FGVI | Stereoscopic video generation · User study | Temporal Consistency: 4.74 (1.33) -> 5.15 (1.22) | Helped |
| Use SVG instead of E2FGVI | Stereoscopic video generation · User study | Image Quality: 4.42 (1.27) -> 5.12 (1.33) | Helped |
| Use SVG instead of E2FGVI | Stereoscopic video generation · User study | Overall Experience: 4.67 (1.04) -> 5.35 (0.99) | Helped |
| Use SVG instead of the strongest inpainting baseline | Stereoscopic video generation · Left-right view semantic consistency | CLIP: 95.29 -> 96.44 | Helped |
| Add frame matrix denoising (ablation) | SVG · Left-right view semantic consistency | CLIP: 95.81 -> 96.44 | Helped |
| Add disocclusion boundary re-injection (ablation) | SVG · Left-right view semantic consistency | CLIP: 95.60 -> 96.44 | Helped |
| Use SVG instead of Deep3D | Stereoscopic video generation · Deep3D comparison user study | Stereo Effect: 2.29 (1.63) -> 5.29 (1.09) | Helped |
| Use SVG instead of Deep3D | Stereoscopic video generation · Deep3D comparison user study | Temporal Consistency: 5.37 (1.23) -> 5.06 (1.25) | Hurt |
| Use SVG instead of Deep3D | Stereoscopic video generation · Deep3D comparison user study | Overall Experience: 3.68 (1.36) -> 5.08 (1.09) | Helped |
| Increase the stereo baseline | SVG · Different stereo baselines | CLIP | Hurt |
Surprising findings
- Deep3D, trained on 3D movies, was rated higher on temporal consistency and image quality than the proposed method in the supplementary study.
Limitations
- Only short videos (16 frames) can be generated due to the Zeroscope architecture.
- Depends on the depth estimation model, which may fail on thin structures.
- Stereo baseline is limited to roughly 20cm before failure.
Experimental setup
Datasets: videos generated by Sora, Lumiere, WALT, and Zeroscope
Benchmarks: user study on generated stereo videos (20 participants, 7-point Likert) CLIP semantic consistency between left and right views
Metrics: stereo effect temporal consistency image quality overall experience CLIP similarity score
Baselines: ProPainter E2FGVI RoDynRF DynIBaR Deep3D
Builds on: RePaint Zeroscope DDPM ProPainter DynIBaR
Closest papers by meaning
- SpatialDreamer: Self-supervised Stereo Video Synthesis from Monocular Input (CVPR 2025 · similarity 0.86)
- SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency (ICLR 2025 · similarity 0.86)
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.85)
- GeoVideo: Introducing Geometric Regularization into Video Generation Model (NeurIPS 2025 · similarity 0.85)
- GenFusion: Closing the Loop between Reconstruction and Generation via Videos (CVPR 2025 · similarity 0.85)
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.