Research map / 3D vision / 3D generation
SPAD: Spatially Aware Multi-View Diffusers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SPAD fine-tunes Stable Diffusion v1.5 into a multi-view diffusion model using epipolar attention and Plücker ray embeddings, enabling consistent multi-view generation from text or a single image with full camera control.
Problem
Generating 3D-consistent multi-view images from text or a single image with precise, arbitrary camera control; existing multi-view diffusion models suffer from content copying between nearby views, flipped predictions, and fixed-viewpoint limits.
Approach
Repurpose a pretrained text-to-image diffusion model by making its self-attention cross-view but restricted to epipolar lines, and inject Plücker ray embeddings as positional encodings so that spatial proximity and ray direction are encoded, which fixes content copying and view flipping.
Key results
- On image-conditioned novel view synthesis on GSO, SPAD achieves PSNR 17.99, SSIM 0.83, LPIPS 0.169, outperforming Zero-1-to-3 (16.10 PSNR) and SyncDreamer (17.18 PSNR) though iNVS has higher PSNR (18.53).
- On Objaverse novel view synthesis, SPAD reaches LPIPS 0.166 and PSNR 20.29, the best LPIPS, versus iNVS PSNR 20.52.
- On text-conditioned generation, SPAD obtains CLIP-score 29.87 versus 28.52 for vanilla MV-DM and 27.76 for SyncDreamer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Epipolar Attention to Vanilla MV-DM (ablation) | MV-DM · Objaverse novel view synthesis | PSNR: 17.56 -> 18.90 | Helped |
| Add Epipolar Attention to Vanilla MV-DM (ablation) | MV-DM · GSO novel view synthesis | PSNR: 15.98 -> 17.13 | Helped |
| Add Plücker Embedding to Vanilla MV-DM (ablation) | MV-DM · Objaverse novel view synthesis | PSNR: 17.56 -> 17.98 | Helped |
| Add Plücker Embedding to Vanilla MV-DM (ablation) | MV-DM · GSO novel view synthesis | PSNR: 15.98 -> 16.15 | Helped |
| Add Plücker Embedding to Epipolar Attention (ablation) | MV-DM (Epipolar) · Objaverse novel view synthesis | LPIPS: 0.19 -> 0.166 | Helped |
| Add Plücker Embedding to Epipolar Attention (ablation) | MV-DM (Epipolar) · GSO novel view synthesis | LPIPS: 0.19 -> 0.169 | Helped |
| Use SPAD instead of Vanilla MV-DM | Text-conditioned generation · Objaverse | CLIP-score: 28.52±3.69 -> 29.87±3.33 | Helped |
| Use SPAD instead of Vanilla MV-DM | Text-conditioned generation · Objaverse | IS: 11.04±0.81 -> 11.18±0.97 | Helped |
| Use SPAD instead of SyncDreamer | Image-conditioned novel view synthesis · Objaverse | LPIPS: 0.174 -> 0.166 | Helped |
| Use SPAD instead of SyncDreamer | Image-conditioned novel view synthesis · GSO | LPIPS: 0.178 -> 0.169 | Helped |
| Use SPAD instead of iNVS | Image-conditioned novel view synthesis · Objaverse | PSNR: 20.52 -> 20.29 | Hurt |
| Use SPAD instead of iNVS | Image-conditioned novel view synthesis · GSO | PSNR: 18.53 -> 17.99 | Hurt |
Surprising findings
- Naive multi-view self-attention (vanilla MV-DM) performs worse than single-component variants on novel view synthesis, causing content copying between views.
- iNVS achieves the highest PSNR on both datasets despite worse SSIM and LPIPS, because it copies pixels directly from the source view.
Limitations
- Built on Stable Diffusion v1.5 with lossy latent compression for image conditioning; authors suggest SDXL could help.
- Relies on epipolar lines that are imprecise without depth; monocular depth could improve correspondences but is not used.
- Evaluated mainly on 2D metrics for text generation, which do not measure multi-view 3D consistency.
Experimental setup
Datasets: Objaverse Cap3D captions Google Scanned Objects
Benchmarks: Objaverse (1,000 captions, text-to-multiview) Objaverse unseen objects (novel view synthesis) Google Scanned Objects
Metrics: CLIP-score Inception Score FID PSNR SSIM LPIPS
Baselines: MVDream SyncDreamer Zero-1-to-3 iNVS Vanilla MV-DM MV-DM (Epipolar) MV-DM (Plücker)
Builds on: Stable Diffusion Epipolar Feature Transformer MVDream ControlNet Instant3D
Closest papers by meaning
- Enhancing Novel View Synthesis via Geometry Grounded Set Diffusion (NeurIPS 2026 · similarity 0.86)
- Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention (NeurIPS 2024 · similarity 0.86)
- WildCAT3D: Appearance-Aware Multi-View Diffusion in the Wild (NeurIPS 2025 · similarity 0.86)
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.86)
- MVDream: Multi-view Diffusion for 3D Generation (ICLR 2024 · similarity 0.86)
- EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.