Research map / 3D vision / 3D generation
Learning 3D reconstruction from images
Reconstructs 3D structure from single or multiple views with learned priors and foundation-model features.
196 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in 3D generation, keyword cluster “semantic · occupancy · driving”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
70 papers in 2023–24 and 79 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 15.5% to 9.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Volumetric and Multi-View CNNs for Object Classification on 3D Data (CVPR 2016 · 1,610 citations)
- 2017 · Latent Multi-View Subspace Clustering (CVPR 2017 · 573 citations)
- 2018 · Triplet-Center Loss for Multi-View 3D Object Retrieval (CVPR 2018 · 385 citations)
- 2019 · DeepVoxels: Learning Persistent 3D Feature Embeddings (CVPR 2019 · 601 citations)
- 2020 · SynSin: End-to-End View Synthesis From a Single Image (CVPR 2020 · 434 citations)
- 2021 · 3D Human Action Representation Learning via Cross-View Consistency Pursuit (CVPR 2021 · 233 citations)
- 2022 · 3D Common Corruptions and Data Augmentation (CVPR 2022 · 89 citations)
- 2023 · Panoptic Lifting for 3D Scene Understanding With Neural Fields (CVPR 2023 · 125 citations)
- 2024 · Probing the 3D Awareness of Visual Foundation Models (CVPR 2024 · 44 citations)
- 2025 · Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels (CVPR 2025 · 15 citations)
- 2026 · LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning (ICML 2026 · 1 citation)
Earliest papers here
- Volumetric and Multi-View CNNs for Object Classification on 3D Data (CVPR 2016 · 1,610 citations)
- Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision (NeurIPS 2016 · 311 citations)
- BoxCars: 3D Boxes as CNN Input for Improved Fine-Grained Vehicle Recognition (CVPR 2016 · 172 citations)
What has been tried
150 papers here report 1,522 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 5%, hurt 19%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add hierarchical modeling to the contrastive learning method | Replica hierarchical segmentation Level 2 · mIoU (%) | 80.4 -> 88.9 | Helped | OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learning (CVPR 2024 · 43 citations) |
| Add intra-view contrastive learning | Land use clustering and region popularity prediction · Performance | Helped | Urban Region Embedding via Multi-View Contrastive Prediction (AAAI 2024 · 35 citations) | |
| Add under-segment mask filtering | ScanNet200 · AP | 10.0 -> 11.0 | Helped | MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation (CVPR 2024 · 32 citations) |
| Add MCP | E-MNIST classification · ACCcls | 93.12 -> 98.37 | Helped | Rethinking Multi-view Representation Learning via Distilled Disentangling (CVPR 2024 · 29 citations) |
| Use iterative diffusion rather than single-step inference | KITTI val · AP3D Moderate | 18.25 -> 22.02 | Helped | MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models (CVPR 2024 · 28 citations) |
| use multi-source embedding (probabilistic 3D grid, 5-DoF pose, semantic 2D map) instead of probabilistic 3D grid alone | Houses3K Test Set · Mean AUC | 81.06% -> 91.19% | Helped | GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction (CVPR 2024 · 27 citations) |
| add the visual and depth encoders to the decoder-only model | Mono3DRefer test set, Overall · Acc@0.25 | 47.31 -> 60.21 | Helped | Mono3DVG: 3D Visual Grounding in Monocular Images (AAAI 2024 · 24 citations) |
| add FRM | CVUSA · R@1 | 94.10% -> 96.70% | Helped | Aligning Geometric Spatial Layout in Cross-View Geo-Localization via Feature Recombination (AAAI 2024 · 23 citations) |
| add foundation-model feature learning | Oxford Radar RobotCar · position error (m) | 6.64 -> 4.80 | Helped | DiffLoc: Diffusion Model for Outdoor LiDAR Localization (CVPR 2024 · 22 citations) |
| add a temporal branch with STT to the appearance branch | Gait3D · Rank-1 | 42.90 -> 48.90 | Helped | HybridGait: A Benchmark for Spatial-Temporal Cloth-Changing Gait Recognition with Hybrid Explorations (AAAI 2024 · 21 citations) |
| replace the DINO ViT-G features with distilled ViT-S features | KeypointNet (airplane, chair, table) · IoU | No clear effect | Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features (CVPR 2024 · 16 citations) | |
| Add GazeFollow with pseudo-3D gaze labels using ST-WSGE to Gaze360 training | Gaze360 Full · angular error (°) | 13.64 -> 13.19 | Helped | Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels (CVPR 2025 · 15 citations) |
What reviewers will expect
Datasets and baselines named in the 43 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ScanNet++ · 10 of 43 papers
- MegaDepth · 6 of 43 papers
- Objaverse · 5 of 43 papers
- ARKitScenes · 5 of 43 papers
- BlendedMVS · 4 of 43 papers
- Gaze360 · 3 of 43 papers
- MegaScenes · 3 of 43 papers
- ScanNetV2 · 3 of 43 papers
Baselines
- DUSt3R · 6 of 43 papers
- MASt3R · 6 of 43 papers
- LoFTR · 4 of 43 papers
- CUT3R · 3 of 43 papers
- RoMA · 3 of 43 papers
- DINOv2 · 3 of 43 papers
- C2F-ARM-BC · 2 of 43 papers
- PerAct · 2 of 43 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 149 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 8 papers
- Google · 7 papers
- Zhejiang University · 6 papers
- The Chinese University of Hong Kong · 6 papers
- Technical University of Munich · 5 papers
- Peking University · 5 papers
- Chinese Academy of Sciences · 5 papers
- Nanyang Technological University · 4 papers
- University of Michigan · 4 papers
- NVIDIA · 4 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Semantic scene completion (153 papers)
- 2D-to-3D pretrained representations (94 papers)
- 3D perception for autonomous driving (73 papers)
- Vision-based 3D occupancy prediction (50 papers)
Elsewhere in the topic
- Diffusion-based novel view synthesis (156 papers)
- Generative indoor scene synthesis (131 papers)
- Hand-object interaction generation (116 papers)
- Feed-forward 3D asset generation and editing (116 papers)
- Score distillation for text-to-3D (106 papers)
- Open-vocabulary 3D scene understanding (95 papers)
- Monocular 3D object reconstruction (94 papers)
- Scene-aware human motion synthesis (82 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).