Research map / 3D vision / 3D generation
Diffusion-based novel view synthesis
Generates consistent novel views from one or few images using multi-view and video diffusion priors.
156 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in 3D generation, keyword cluster “3d generation · text · consistent”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Leveling off Several papers from 2025–26 still report on 1 tracked benchmark, but the best result has barely moved since 2024.
37 papers in 2023–24 and 112 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 8.2% to 12.7%.
Open questions the data points to
- Papers report SyncDreamer on GSO (LPIPS) anywhere from 0.146 to 0.198: comparisons against it depend on whose number you take.
What looks settled
- GSO (CD): best up to 2024 0.04, best in 2025–26 0.101 from 2 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Transformation-Grounded Image Generation Network for Novel 3D View Synthesis (CVPR 2017 · 293 citations)
- 2020 · Single-View View Synthesis With Multiplane Images (CVPR 2020 · 315 citations)
- 2021 · Sketch2Model: View-Aware 3D Modeling From Single Free-Hand Sketches (CVPR 2021 · 93 citations)
- 2022 · Multi-View Consistent Generative Adversarial Networks for 3D-Aware Image Synthesis (CVPR 2022 · 41 citations)
- 2023 · SparseFusion: Distilling View-Conditioned Diffusion for 3D Reconstruction (CVPR 2023 · 170 citations)
- 2024 · ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image (CVPR 2024 · 59 citations)
- 2025 · GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (CVPR 2025 · 49 citations)
- 2026 · DeOcc-1-to-3: 3D De-Occlusion from a Single Image via Self-Supervised Multi-View Diffusion (AAAI 2026 · 4 citations)
Earliest papers here
- Transformation-Grounded Image Generation Network for Novel 3D View Synthesis (CVPR 2017 · 293 citations)
- Single-View View Synthesis With Multiplane Images (CVPR 2020 · 315 citations)
- Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis (CVPR 2020 · 112 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GSO · PSNR (higher is better)
6 papers, 2024–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 23.57 | GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation (NeurIPS 2024 · 3 citations) |
GSO · LPIPS (lower is better)
6 papers, 2024–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.0619 | Free3D: Consistent Novel View Synthesis without 3D Representation (CVPR 2024 · 34 citations) |
GSO · SSIM (higher is better)
5 papers, 2024–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.872 | GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation (NeurIPS 2024 · 3 citations) |
GSO · CD (lower is better)
4 papers, 2024–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 0.04 | DMV3D: Denoising Multi-view Diffusion Using 3D Large Reconstruction Model (ICLR 2024) |
What has been tried
87 papers here report 879 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 4%, hurt 20%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Train using all three datasets rather than omitting any one dataset | DTU · LPIPS | 0.421 vs 0.446; 0.456; 0.435 | Helped | ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image (CVPR 2024 · 59 citations) |
| use independent multi-view sampling without sliding-window overlap instead of alternating sampling | NSFF · PSNR | 22.15 -> 20.27 | Hurt | CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models (CVPR 2025 · 34 citations) |
| Add ray embeddings to the input | Objaverse · LPIPS | 0.0938 -> 0.0858 | Helped | Free3D: Consistent Novel View Synthesis without 3D Representation (CVPR 2024 · 34 citations) |
| Add real MvImgNet data to synthetic Objaverse training | Unseen Evaluation · PSNR | 15.5 -> 19.0 | Helped | LRM: Large Reconstruction Model for Single Image to 3D (ICLR 2024 · 31 citations) |
| plug ConsistNet into decoder only instead of encoder only | Google Scanned Objects · LPIPS Alex | 0.14 -> 0.11 | Helped | ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion (CVPR 2024 · 30 citations) |
| Use F = 8 instead of F = 4 neighboring views | GSO, elevation 30° · PSNR | 20.49 -> 20.27 | Hurt | EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion (CVPR 2024 · 28 citations) |
| Use DINO feature cosine similarity instead of RGB PSNR/SSIM | RealEstate10K test · consistency alignment | Helped | MET3R: Measuring Multi-View Consistency in Generated Images (CVPR 2025 · 25 citations) | |
| Add appearance diffusion to Masked NeRF | Real-S · LPIPS | 0.761 -> 0.587 | Helped | MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior (CVPR 2024 · 22 citations) |
| Add 1D multi-view attention | THuman2.1 · PSNR | 22.84 -> 23.94 | Helped | MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement (AAAI 2025 · 20 citations) |
| Add the ControlNet camera-conditioning branch to a LoRA-only branch | RealEstate10K · FID | 19.02 -> 18.75 | Helped | Wonderland: Navigating 3D Scenes from a Single Image (CVPR 2025 · 19 citations) |
| add image and video priors instead of training from scratch | RealEstate10K test, 256px · FID (long-term) | 85.30 -> 30.15 | Helped | MultiDiff: Consistent Novel View Synthesis from a Single Image (CVPR 2024 · 19 citations) |
| Keep λvideo at 1.0 through timestep 50 instead of decaying to 0.5 | GSO, azimuth range 30–45 degrees · FOR8 | 0.2923 -> 0.2958 | Hurt | ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models (CVPR 2024 · 19 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| SyncDreamer | GSO | LPIPS | 0.146 – 0.198 | 3 papers, e.g. CVPR 2024 and ICML 2025 |
| SyncDreamer | GSO | SSIM | 0.739 – 0.798 | 3 papers, e.g. ICML 2025 and CVPR 2024 |
| SyncDreamer | GSO | PSNR | 18.75 – 20.05 | 3 papers, e.g. ICML 2025 and CVPR 2024 |
| SyncDreamer | GSO novel-view synthesis | LPIPS | 0.14 – 0.178 | 2 papers, e.g. CVPR 2024 and CVPR 2024 |
| Zero123-XL | RTMV | SSIM | 0.52 – 0.595 | 2 papers, e.g. CVPR 2024 and NeurIPS 2024 |
What reviewers will expect
Datasets and baselines named in the 55 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- RealEstate10K · 14 of 55 papers
- Objaverse · 7 of 55 papers
- DL3DV · 6 of 55 papers
- CO3D · 5 of 55 papers
- GSO · 5 of 55 papers
- DyCheck · 4 of 55 papers
- ScanNet++ · 4 of 55 papers
- Waymo Open Dataset · 4 of 55 papers
Baselines
- ViewCrafter · 7 of 55 papers
- MVSplat · 6 of 55 papers
- ZeroNVS · 5 of 55 papers
- MotionCtrl · 5 of 55 papers
- 3DGS · 5 of 55 papers
- SyncDreamer · 4 of 55 papers
- PixelSplat · 4 of 55 papers
- Wonder3D · 3 of 55 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 149 papers; from OpenAlex and the papers' affiliations).
- Google · 6 papers
- Peking University · 5 papers
- Zhejiang University · 5 papers
- University of Toronto · 5 papers
- Tencent · 4 papers
- Tsinghua University · 4 papers
- Meta · 4 papers
- Stanford University · 3 papers
- Technical University of Munich · 3 papers
- Carnegie Mellon University · 3 papers
Use it
Download BibTeX of 31 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Score distillation for text-to-3D (106 papers)
- Feed-forward 3D asset generation and editing (116 papers)
- Text-to-3D scene generation (58 papers)
Elsewhere in the topic
- Learning 3D reconstruction from images (196 papers)
- Semantic scene completion (153 papers)
- Generative indoor scene synthesis (131 papers)
- Hand-object interaction generation (116 papers)
- Open-vocabulary 3D scene understanding (95 papers)
- 2D-to-3D pretrained representations (94 papers)
- Monocular 3D object reconstruction (94 papers)
- Scene-aware human motion synthesis (82 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).