Research map / 3D vision / 3D generation
EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EpiDiff, which inserts a lightweight epipolar-constrained attention block into a frozen Zero123 UNet to generate consistent multiview images from one view in about 12 seconds.
Problem
Single-image multiview synthesis for 3D generation lacks cross-view consistency; global 3D feature volume methods like SyncDreamer are slow, generalize poorly, and yield blurry or distorted images.
Approach
Localized epipolar attention: each target-view ray attends only to features from the F nearest views along epipolar lines, inserted as a trainable add-on into a frozen 2D NVS diffusion UNet, with Plücker-coordinate and ray-relative positional encodings.
Key results
- At fixed 30° elevation on GSO, EpiDiff reaches 20.49 PSNR, 0.855 SSIM, 0.128 LPIPS in 12s, versus SyncDreamer's 20.11 / 0.829 / 0.159 in 60s.
- Under uniform elevations, EpiDiff gets 18.83 PSNR vs 15.91 for Zero123 and 15.90 for SyncDreamer.
- Single-view reconstruction Chamfer Distance 0.0429 and Volume IoU 0.4518, the best among the compared methods (SyncDreamer 0.0496 / 0.4149).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EpiDiff instead of SyncDreamer | Zero123-based multiview synthesis · GSO, elevation 30° | PSNR: 20.11 -> 20.49 | Helped |
| Use EpiDiff instead of SyncDreamer | Zero123-based multiview synthesis · GSO, elevation 30° | Runtime: 60s -> 12s | Helped |
| Use EpiDiff instead of SyncDreamer | Zero123-based multiview synthesis · GSO, uniform elevations | PSNR: 15.90 -> 18.83 | Helped |
| Use EpiDiff instead of SyncDreamer | Zero123-based multiview synthesis · GSO surface reconstruction | Volume IoU: 0.4149 -> 0.4518 | Helped |
| Use F = 8 instead of F = 4 neighboring views (ablation) | EpiDiff · GSO, elevation 30° | PSNR: 20.49 -> 20.27 | Hurt |
| Use F = 16 instead of F = 4 neighboring views (ablation) | EpiDiff · GSO, elevation 30° | PSNR: 20.49 -> 20.32 | Hurt |
| Remove light field positional encoding (ablation) | EpiDiff · GSO, elevation 30° | PSNR: 20.49 -> 19.83 | Hurt |
| Remove ray-relative positional encoding (ablation) | EpiDiff · GSO, elevation 30° | PSNR: 20.49 -> 20.01 | Hurt |
Surprising findings
- Using more neighboring views (F=8 or 16) did not improve quality and increased runtime, so localized aggregation with F=4 worked best.
Limitations
- Performance degrades for novel views far from the input view, due to instability of the base NVS model.
- Multiview synthesis and 3D reconstruction remain two separate stages rather than a unified pipeline.
- Reconstruction from fixed-elevation views produces protruding bottoms because the bottom is not visible.
Experimental setup
Datasets: Objaverse LVIS subset Google Scanned Objects
Benchmarks: Google Scanned Objects
Metrics: PSNR SSIM LPIPS Chamfer Distance Volume IoU runtime
Baselines: Zero123 SyncDreamer One-2-3-45 Point-E Shap-E
Builds on: Zero123 Latent Diffusion Models Light Field Networks GPNR
Closest papers by meaning
- Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention (NeurIPS 2024 · similarity 0.90)
- Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views (AAAI 2024 · similarity 0.88)
- MultiDiff: Consistent Novel View Synthesis from a Single Image (CVPR 2024 · similarity 0.88)
- MVDream: Multi-view Diffusion for 3D Generation (ICLR 2024 · similarity 0.88)
- Efficient-3Dim: Learning a Generalizable Single-image Novel-view Synthesizer in One Day (ICLR 2024 · similarity 0.88)
- Enhancing Novel View Synthesis via Geometry Grounded Set Diffusion (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.