Research map / Image generation / Fast diffusion sampling
ViewFusion: Towards Multi-View Consistency via Interpolated Denoising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free ViewFusion extends pre-trained single-view novel-view diffusion models (Zero-1-to-3) to auto-regressive, multi-view-conditioned generation via interpolated noise denoising, improving multi-view consistency and 3D reconstruction on ABO and GSO.
Problem
Novel-view synthesis with diffusion models generates each view independently from a single reference image, producing multi-view inconsistencies that harm downstream 3D reconstruction.
Approach
Generate views auto-regressively and, at each denoising step, combine the noise predictions conditioned on all known views (initial and previously generated) via a weighted interpolation with distance-based weights, sharing one interpolated noise across the reverse process, so no fine-tuning is needed.
Key results
- On ABO with 16 views, ViewFusion improves SIFT matches to 13.51 vs 12.36 for SyncDreamer and LPIPS to 0.1602 (lower is better), indicating better adjacent-view consistency.
- On GSO single-view reconstruction, ViewFusion (36 generated views) reaches CD 0.0164 and F-score 0.8436 vs 0.0207 and 0.8078 for Zero-1-to-3 (36 views).
- Multi-view conditioning with 3 condition views improves ABO PSNR from 21.61 (1 view) to 22.74 (3 views).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add interpolated denoising to Zero123 | Zero123 · ABO, free renderings | SSIM: 0.8796 -> 0.8848 | Helped |
| Add interpolated denoising to Zero123 | Zero123 · GSO, free renderings | LPIPS: 0.1029 -> 0.0958 | Helped |
| Add autoregressive interpolated denoising | Zero123 · ABO, 16-view spin video | LPIPS: 0.1782 -> 0.1602 | Helped |
| Add autoregressive interpolated denoising | Zero123 · GSO, 16-view spin video | CLIP similarity: 0.9482 -> 0.9601 | Helped |
| Add more condition views (from 1 to 3) | ViewFusion · ABO | SSIM: 0.8820 -> 0.8995 | Helped |
| Add more condition views (from 1 to 3) | ViewFusion · GSO | SSIM: 0.8721 -> 0.8979 | Helped |
| Use the last generated view as the next condition instead of interpolated denoising (ablation) | ViewFusion · ABO | SSIM: 0.8848 -> 0.8010 | Hurt |
| Interpolate condition-image and pose embeddings instead of denoising (ablation) | ViewFusion · ABO | LPIPS: 0.0923 -> 0.3770 | Hurt |
| Interpolate final image feature maps instead of denoising (ablation) | ViewFusion · ABO | LPIPS: 0.0923 -> 0.1246 | Hurt |
| Use stochastic conditioning instead of interpolated denoising (ablation) | ViewFusion · ABO | SSIM: 0.8848 -> 0.8699 | Hurt |
| Use stochastic conditioning instead of interpolated denoising (ablation) | ViewFusion · GSO | SSIM: 0.8820 -> 0.8658 | Hurt |
Surprising findings
- Stochastic conditioning from prior work fails when applied to Zero-1-to-3, which the authors attribute to category-specific training of the original model.
- Interpolated outputs achieves the highest SSIM and PSNR on ABO yet produces blurred images, as shown by worse LPIPS.
Limitations
- Generating views auto-regressively requires multiple sequential steps per target view, which increases inference cost.
- Evaluation is restricted to out-of-distribution object datasets (ABO, GSO) and fixed or arbitrary object-centric views.
Experimental setup
Datasets: ABO GSO
Benchmarks: ABO GSO
Metrics: SSIM PSNR LPIPS SIFT matching count CLIP similarity Chamfer distance F-score
Baselines: Zero-1-to-3 SyncDreamer One-2-3-45 Point-E Shap-E MonoSDF Stochastic conditioning
Builds on: Zero-1-to-3 DDPM Stochastic conditioning NeuS
Closest papers by meaning
- Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.89)
- MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model (CVPR 2025 · similarity 0.88)
- GenFusion: Closing the Loop between Reconstruction and Generation via Videos (CVPR 2025 · similarity 0.88)
- UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis (NeurIPS 2025 · similarity 0.88)
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.88)
- SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Model (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.