Research map / 3D vision / 3D generation
ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ViDAR, which personalises an SDXL diffusion model (DreamBooth LoRA) to a monocular video, uses it to enhance degraded novel-view renders of a MoSca reconstruction, and retrains with dynamic-region-masked losses and sampled camera pose optimisation.
Problem
Monocular dynamic novel view synthesis is ill-posed, with scarce supervision for novel viewpoints far from the input trajectory, and diffusion-generated pseudo-views are spatio-temporally inconsistent.
Approach
Use a scene-personalised SDXL (DreamBooth-LoRA) to refine rendered novel views into pseudo-multi-view supervision, apply the diffusion supervision only to masked dynamic regions, and jointly optimise the sampled camera poses so pseudo-GTs align with scene geometry.
Key results
- On DyCheck half-res co-visibility masked metrics, ViDAR reports PSNR-m 19.69 and SSIM-m 0.7126, above MoSca (19.32, 0.7060).
- On dynamic-region metrics (half-res), ViDAR reaches PSNR-D 16.46 vs MoSca 15.63, a gain of about 0.8 dB.
- Ablation: removing sampled camera optimisation drops PSNR-m from 19.00 to 18.39, and removing dynamic reconstruction drops SSIM-m from 0.6672 to 0.6274.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ViDAR instead of MoSca | DyCheck co-visibility Half-Res · DyCheck co-visibility Half-Res | PSNR-m: 19.32 -> 19.69 | Helped |
| Use ViDAR instead of MoSca | DyCheck dynamic Half-Res · DyCheck dynamic Half-Res | PSNR-D: 15.63 -> 16.46 | Helped |
| Use ViDAR instead of MoSca | DyCheck co-visibility Full-Res · DyCheck co-visibility Full-Res | SSIM-m: 0.6560 -> 0.6672 | Helped |
| Use ViDAR instead of CAT4D | DyCheck co-visibility SoM-5 · DyCheck co-visibility SoM-5 | LPIPS-m: 0.3410 -> 0.3774 | Hurt |
| Add tracking-based Gaussian classification (ablation) | ViDAR · DyCheck co-visibility Full-Res | PSNR-m: 18.88 -> 19.00 | Helped |
| Add sampled camera optimisation (ablation) | ViDAR · DyCheck co-visibility Full-Res | PSNR-m: 18.39 -> 19.00 | Helped |
| Add dynamic reconstruction (ablation) | ViDAR · DyCheck co-visibility Full-Res | PSNR-m: 18.93 -> 19.00 | Helped |
| Add sampled camera optimisation, dynamic reconstruction, and tracking-based Gaussian classification (ablation) | ViDAR · DyCheck co-visibility Full-Res | PSNR-m: 18.46 -> 19.00 | Helped |
Surprising findings
- CAT4D achieves a lower LPIPS than ViDAR on SoM-5 co-visibility metrics, which the authors attribute to reduced spatio-temporal consistency.
Limitations
- Diffusion only enhances renders, so it is bounded by the accuracy of the initial 4D reconstruction and cannot repair major geometric artefacts.
- Diffusion-generated supervision is spatio-temporally inconsistent, requiring masking and pose optimisation.
- Evaluation relies mainly on DyCheck, which is the only widely used benchmark judged appropriate for this setting.
Experimental setup
Datasets: DyCheck (iPhone)
Benchmarks: DyCheck (iPhone)
Metrics: PSNR SSIM LPIPS PSNR-m SSIM-m LPIPS-m PSNR-D SSIM-D LPIPS-D
Baselines: MoSca Shape of Motion CAT4D Gaussian Marbles SoM 4DGS T-NeRF Nerfies HyperNeRF DynPoint
Builds on: MoSca DreamBooth Stable Diffusion XL ReconFusion Track Anything
Closest papers by meaning
- Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos (NeurIPS 2025 · similarity 0.89)
- CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models (CVPR 2025 · similarity 0.89)
- ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models (CVPR 2024 · similarity 0.89)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.88)
- NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer (ICLR 2025 · similarity 0.88)
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.