Research map / 3D vision / 3D generation
Vision-based 3D occupancy prediction
Predicts semantic occupancy with efficient representations such as octrees, superquadrics and low-rank decomposition.
50 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in 3D generation, keyword cluster “semantic · occupancy · driving”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 1 tracked benchmark.
10 papers in 2023–24 and 37 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 2.2% to 4.2%.
Open questions the data points to
- Occ3D-nuScenes (mIoU): best went from 46.2 (up to 2024) to 51.66 in 2025 — SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction (CVPR 2025 · 0 citations)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · OccuSeg: Occupancy-Aware 3D Instance Segmentation (CVPR 2020 · 260 citations)
- 2022 · STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes (CVPR 2022 · 50 citations)
- 2023 · Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction (CVPR 2023 · 307 citations)
- 2024 · COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction (CVPR 2024 · 41 citations)
- 2025 · GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Prediction (CVPR 2025 · 18 citations)
- 2026 · Vision-Only Gaussian Splatting for Collaborative Semantic Occupancy Prediction (AAAI 2026 · 1 citation)
Earliest papers here
- OccuSeg: Occupancy-Aware 3D Instance Segmentation (CVPR 2020 · 260 citations)
- ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings (CVPR 2020 · 118 citations)
- STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes (CVPR 2022 · 50 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
Occ3D-nuScenes · mIoU (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 46.2 | COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction (CVPR 2024 · 41 citations) |
| 2025 | 51.66 | SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction (CVPR 2025 · 0 citations) |
What has been tried
32 papers here report 385 measured changes (a component added, removed or swapped, compared with a baseline): helped 76%, no clear effect 4%, hurt 18%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add geometry-aware occupancy encoder | Occ3D-nuScenes · IoU | 70.36 -> 70.89 | Helped | COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction (CVPR 2024 · 41 citations) |
| Remove the 3D flow prediction head | nuScenes, inflated GMO · IoUf (2.0 s) | 23.89 -> 22.99 | Hurt | Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications (CVPR 2024 · 32 citations) |
| add the Doppler bins descriptor instead of average-pooling the Doppler axis | K-Radar well-condition test, 12.8m · IoU (%) | 48.1 -> 48.8 | Helped | RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radar (NeurIPS 2024 · 28 citations) |
| Use global hardness only | SemanticKITTI val, long range (L) · mIoU | 13.33 -> 13.30 | No clear effect | Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation (CVPR 2024 · 27 citations) |
| add probabilistic Gaussian modeling | nuScenes val · mIoU | 16.00 -> 19.61 | Helped | GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Prediction (CVPR 2025 · 18 citations) |
| add matrix encoding to vector encoding | SemanticKITTI validation · IoU | 30.01 -> 37.85 | Helped | LowRankOcc: Tensor Decomposition and Low-Rank Recovery for Vision-based 3D Semantic Occupancy Prediction (CVPR 2024 · 16 citations) |
| Use octree queries instead of dense queries | Occ3D-nuScenes validation, reduced input size 0.3x · mIoU | 34.17 -> 34.91 | Helped | OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries (NeurIPS 2024 · 15 citations) |
| Add probability/logit distillation to rendering distillation | Occ3D · mIoU | 37.98 -> 38.53 | Helped | RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation (AAAI 2024 · 14 citations) |
| omit ego-motion alignment | nuScenes validation set · mIoU | 3.0% decrease | Hurt | GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction (CVPR 2025 · 13 citations) |
| add the octree structure to the non-octree variant | nuScenes validation · mIoU | +2.01 | Helped | OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · 11 citations) |
| Add Metric3D geometric alignment to feature alignment | Occ3D-nuScenes · IoU | 43.87 -> 45.19 | Helped | GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding (CVPR 2025 · 6 citations) |
| Add OA-SCA | OpenOcc ablation · RayIoU | 35.1 -> 35.7 | Helped | STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Prediction (CVPR 2025 · 6 citations) |
What reviewers will expect
Datasets and baselines named in the 22 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- nuScenes · 13 of 22 papers
- Occ3D-nuScenes · 10 of 22 papers
- SemanticKITTI · 3 of 22 papers
- SSCBench-KITTI360 · 2 of 22 papers
- KITTI-360 · 2 of 22 papers
- Occ3D · 2 of 22 papers
- SurroundOcc occupancy annotations · 2 of 22 papers
- nuScenes-Occupancy · 2 of 22 papers
Baselines
- SurroundOcc · 11 of 22 papers
- BEVFormer · 9 of 22 papers
- TPVFormer · 9 of 22 papers
- OccFormer · 8 of 22 papers
- MonoScene · 7 of 22 papers
- FB-Occ · 6 of 22 papers
- COTR · 6 of 22 papers
- SparseOcc · 5 of 22 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 47 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 7 papers
- Huawei · 3 papers
- National University of Defense Technology · 2 papers
- Xi’an Jiaotong University · 2 papers
- PhiGent Robotics · 2 papers
- Horizon Robotics · 2 papers
- University of Science and Technology Beijing · 2 papers
- ShanghaiTech University · 2 papers
Use it
Download BibTeX of 25 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Learning 3D reconstruction from images (196 papers)
- Semantic scene completion (153 papers)
- 2D-to-3D pretrained representations (94 papers)
- 3D perception for autonomous driving (73 papers)
Elsewhere in the topic
- Diffusion-based novel view synthesis (156 papers)
- Generative indoor scene synthesis (131 papers)
- Hand-object interaction generation (116 papers)
- Feed-forward 3D asset generation and editing (116 papers)
- Score distillation for text-to-3D (106 papers)
- Open-vocabulary 3D scene understanding (95 papers)
- Monocular 3D object reconstruction (94 papers)
- Scene-aware human motion synthesis (82 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).