Research map / 3D vision / 3D generation
Semantic scene completion
Predicts complete semantic 3D scenes from monocular images using foundation-model guidance and scene representations.
153 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in 3D generation, keyword cluster “semantic · occupancy · driving”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 5 tracked benchmarks.
35 papers in 2023–24 and 79 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 7.7% to 9.0%.
Open questions the data points to
- SemanticKITTI hidden test (mIoU): best went from 14.6 (up to 2024) to 17.7 in 2025 — Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance (NeurIPS 2025 · 0 citations)
What looks settled
- SSCBench-KITTI-360 test (mIoU): best up to 2024 22.39, best in 2025–26 20.81 from 5 papers, at most 0.2% better, if at all.
- SSCBench-KITTI-360 test (IoU): best up to 2024 48.07, best in 2025–26 46.35 from 3 papers, at most 0.2% better, if at all.
- SemanticKITTI val (mIoU): best up to 2024 30.11, best in 2025–26 26.51 from 3 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Attend, Infer, Repeat: Fast Scene Understanding with Generative Models (NeurIPS 2016 · 242 citations)
- 2017 · Neural Scene De-Rendering (CVPR 2017 · 110 citations)
- 2018 · ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans (CVPR 2018 · 329 citations)
- 2019 · SceneCode: Monocular Dense Semantic Reconstruction Using Learned Encoded Scene Representations (CVPR 2019 · 74 citations)
- 2020 · Mapillary Street-Level Sequences: A Dataset for Lifelong Place Recognition (CVPR 2020 · 227 citations)
- 2021 · Weakly Supervised Learning of Rigid 3D Scene Flow (CVPR 2021 · 91 citations)
- 2022 · NICE-SLAM: Neural Implicit Scalable Encoding for SLAM (CVPR 2022 · 812 citations)
- 2023 · VoxFormer: Sparse Voxel Transformer for Camera-Based 3D Semantic Scene Completion (CVPR 2023 · 244 citations)
- 2024 · Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding (CVPR 2024 · 80 citations)
- 2025 · Continuous 3D Perception Model with Persistent State (CVPR 2025 · 81 citations)
- 2026 · EAGLE: Episodic Appearance- and Geometry-aware Memory for Unified 2D-3D Visual Query Localization in Egocentric Vision (AAAI 2026 · 2 citations)
Earliest papers here
- Attend, Infer, Repeat: Fast Scene Understanding with Generative Models (NeurIPS 2016 · 242 citations)
- Neural Scene De-Rendering (CVPR 2017 · 110 citations)
- Deep Supervision With Shape Concepts for Occlusion-Aware 3D Object Parsing (CVPR 2017 · 91 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
SSCBench-KITTI-360 test · mIoU (higher is better)
8 papers, 2024–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 22.39 | PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness (CVPR 2024 · 26 citations) |
SSCBench-KITTI-360 test · IoU (higher is better)
5 papers, 2024–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 48.07 | Context and Geometry Aware Voxel Transformer for Semantic Scene Completion (NeurIPS 2024) |
SemanticKITTI val · mIoU (higher is better)
5 papers, 2024–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 30.11 | PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness (CVPR 2024 · 26 citations) |
SemanticKITTI hidden test · mIoU (higher is better)
4 papers, 2022–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 11.08 | MonoScene: Monocular 3D Semantic Scene Completion (CVPR 2022 · 288 citations) |
| 2024 | 14.6 | H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion (AAAI 2024 · 20 citations) |
| 2025 | 17.7 | Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance (NeurIPS 2025 · 0 citations) |
SemanticKITTI test · mIoU (higher is better)
4 papers, 2024–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 16.63 | Context and Geometry Aware Voxel Transformer for Semantic Scene Completion (NeurIPS 2024) |
What has been tried
104 papers here report 1,140 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 4%, hurt 21%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Revisit images using the final state | 7-Scenes · Acc | 0.126 -> 0.113 | Helped | Continuous 3D Perception Model with Persistent State (CVPR 2025 · 81 citations) |
| add DINO features | Mip-NeRF360 annotated scenes · mAP | 0.676 -> 0.815 | Helped | Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding (CVPR 2024 · 80 citations) |
| Add pre-trained weights to the Image Encoder | SemanticKITTI val · mIoU | 10.44 -> 12.59 (+2.15) | Helped | Symphonize 3D Semantic Scene Completion with Contextual Instance Queries (CVPR 2024 · 60 citations) |
| use the MIMO ensemble instead of a single subnet | Semantic KITTI val · All PQ† | 26.49 -> 31.42 | Helped | PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness (CVPR 2024 · 26 citations) |
| Add the hierarchical pixel decoder | SemanticKITTI val · mIoU | 13.42 -> 13.73 | Helped | H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion (AAAI 2024 · 20 citations) |
| Add a static field rather than use full 4D iNGP without one | Dynamic-32 scene reconstruction, full image · PSNR | 26.55 -> 27.21 | Helped | EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision (ICLR 2024 · 17 citations) |
| Increase the key-point count from Ne=768 to Ne=1536 | RoboRefit testB · Acc@0.5 | 50.64 -> 55.62 | Helped | SUGAR: Pre-training 3D Visual Representations for Robotics (CVPR 2024 · 14 citations) |
| add multi-view 2D features | ScanRefer · Acc@0.25 | 36.0 -> 57.8 | Helped | Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning (CVPR 2025 · 13 citations) |
| add Spatial Sensory Fusion (SSF) | SemanticKITTI val · IoU | 44.38 -> 44.88 | Helped | Bi-SSC: Geometric-Semantic Bidirectional Fusion for Camera-based 3D Semantic Scene Completion (CVPR 2024 · 10 citations) |
| Remove residual connection from point cloud module | ScanNet · mIoU | 66.9 -> 64.6 | Hurt | Memory-based Adapters for Online 3D Scene Perception (CVPR 2024 · 9 citations) |
| Replace the baseline image encoder with RepViT | SemanticKITTI val · mIoU | 15.43 -> 15.59 | Helped | VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion (AAAI 2025 · 8 citations) |
| replace scene-level features with G3Flow semantic flow | Shoe Place (T) · success rate (%) | 67.7±1.5 -> 83.0±3.6 | Helped | G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation (CVPR 2025 · 8 citations) |
What reviewers will expect
Datasets and baselines named in the 39 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ScanNet · 12 of 39 papers
- SemanticKITTI · 8 of 39 papers
- SSCBench-KITTI-360 · 7 of 39 papers
- ARKitScenes · 5 of 39 papers
- RealEstate10K · 4 of 39 papers
- ScanNet200 · 4 of 39 papers
- Replica · 3 of 39 papers
- Hypersim · 3 of 39 papers
Baselines
- MonoScene · 7 of 39 papers
- TPVFormer · 7 of 39 papers
- OccFormer · 7 of 39 papers
- VoxFormer · 5 of 39 papers
- Symphonies · 4 of 39 papers
- SurroundOcc · 4 of 39 papers
- LangSplat · 3 of 39 papers
- Symphonize · 3 of 39 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 114 papers; from OpenAlex and the papers' affiliations).
- Google · 5 papers
- NVIDIA · 4 papers
- National University of Singapore · 4 papers
- Zhejiang University · 4 papers
- Hunan University · 3 papers
- Yonsei University · 3 papers
- University of Toronto · 3 papers
- Vector Institute · 3 papers
- Carnegie Mellon University · 3 papers
- Xi’an Jiaotong University · 3 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Learning 3D reconstruction from images (196 papers)
- 2D-to-3D pretrained representations (94 papers)
- 3D perception for autonomous driving (73 papers)
- Vision-based 3D occupancy prediction (50 papers)
Elsewhere in the topic
- Diffusion-based novel view synthesis (156 papers)
- Generative indoor scene synthesis (131 papers)
- Hand-object interaction generation (116 papers)
- Feed-forward 3D asset generation and editing (116 papers)
- Score distillation for text-to-3D (106 papers)
- Open-vocabulary 3D scene understanding (95 papers)
- Monocular 3D object reconstruction (94 papers)
- Scene-aware human motion synthesis (82 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).