Research map / Detection and segmentation / 3D detection and driving
3D objectness and scene segmentation
Learn open-world 3D objectness and instance segmentation with self-supervised and contrastive methods.
57 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in 3D detection and driving, keyword cluster “3d object · monocular · kitti”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
23 papers in 2023–24 and 15 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 14.6% to 9.9%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · A Continuous Occlusion Model for Road Scene Understanding (CVPR 2016 · 35 citations)
- 2017 · SGM-Nets: Semi-Global Matching With Neural Networks (CVPR 2017 · 295 citations)
- 2018 · Perceiving, Learning, and Recognizing 3D Objects: An Approach to Cognitive Service Robots (AAAI 2018 · 31 citations)
- 2019 · PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding (CVPR 2019 · 700 citations)
- 2020 · Learning to Evaluate Perception Models Using Planner-Centric Metrics (CVPR 2020 · 72 citations)
- 2021 · VIP-DeepLab: Learning Visual Perception With Depth-Aware Video Panoptic Segmentation (CVPR 2021 · 145 citations)
- 2022 · BoxeR: Box-Attention for 2D and 3D Transformers (CVPR 2022 · 39 citations)
- 2023 · LargeKernel3D: Scaling Up Kernels in 3D Sparse CNNs (CVPR 2023 · 135 citations)
- 2024 · UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes (CVPR 2024 · 30 citations)
- 2025 · InstanceGaussian: Appearance-Semantic Joint Gaussian Representation for 3D Instance-Level Perception (CVPR 2025 · 9 citations)
- 2026 · OpenScan: A Benchmark for Generalized Open-Vocabulary 3D Scene Understanding (AAAI 2026 · 2 citations)
Earliest papers here
- A Continuous Occlusion Model for Road Scene Understanding (CVPR 2016 · 35 citations)
- SGM-Nets: Semi-Global Matching With Neural Networks (CVPR 2017 · 295 citations)
- Joint Sequence Learning and Cross-Modality Convolution for 3D Biomedical Segmentation (CVPR 2017 · 189 citations)
What has been tried
48 papers here report 550 measured changes (a component added, removed or swapped, compared with a baseline): helped 80%, no clear effect 4%, hurt 14%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use both color and geometry features instead of geometry features alone for pseudo mask generation | ScanNet · AP Final | 13.3 -> 15.9 | Helped | UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes (CVPR 2024 · 30 citations) |
| add question information | LASO Seen · mIoU | 10.8 -> 20.8 | Helped | LASO: Language-guided Affordance Segmentation on 3D Object (CVPR 2024 · 25 citations) |
| Add the Semantic Network | ScanNetV2 val · mAP | 58.4 -> 59.3 | Helped | Edge-Aware 3D Instance Segmentation Network with Intelligent Semantic Prior (CVPR 2024 · 14 citations) |
| Blend GPT-3 text prompts and Stable Diffusion visual prompts instead of using either alone with Detic-generated boxes | SUN RGB-D · mAP25 | 16.34 -> 18.19; 18.09 -> 18.19 | Helped | FM-OV3D: Foundation Model-Based Cross-Modal Knowledge Blending for Open-Vocabulary 3D Detection (AAAI 2024 · 13 citations) |
| use joint representation and progressive joint training | ScanNet · Semantic Seg. mIoU | 30.71 -> 40.66 | Helped | InstanceGaussian: Appearance-Semantic Joint Gaussian Representation for 3D Instance-Level Perception (CVPR 2025 · 9 citations) |
| Use VFM-assisted semantic superpixels | nuScenes, 1% labels, semantic segmentation · mIoU | 38.2 -> 41.1 | Helped | Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception (CVPR 2024 · 7 citations) |
| Add dense distillation | ScanNetV2 · AP25 | 61.1 -> 62.4 | Helped | SAM-Guided Masked Token Prediction for 3D Scene Understanding (NeurIPS 2024 · 5 citations) |
| Add 3D-Aware 2D Mask Tracking | ScanNet++ class-agnostic 3DIS · AP | 17.0 -> 17.9 | Helped | Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Tracking (CVPR 2025 · 2 citations) |
| Add affordance-head CoT | Unseen Affordance · AUC | 67.18 -> 69.81 | Helped | GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · 2 citations) |
| Add multi-view images to point-cloud input | LASA test, Chair · Chamfer L2 | 5.68 -> 3.57 | Helped | LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset (CVPR 2024 · 2 citations) |
| Add CLGM to the baseline | SemanticKITTI validation · mIoU (%) | 8.1 -> 19.5 | Helped | AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction (AAAI 2025 · 1 citation) |
| Add Context-aware refinement to Surface-aware refinement | WOD training set, Vehicle · AP3D | 30.34 -> 38.65 | Helped | OpenBox: Annotate Any Bounding Boxes in 3D (NeurIPS 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 9 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ScanNet · 3 of 9 papers
- nuScenes · 2 of 9 papers
Baselines
- SAM3D · 2 of 9 papers
- SAI3D · 2 of 9 papers
- MaskClustering · 2 of 9 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 38 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 4 papers
- The Chinese University of Hong Kong · 4 papers
- Meta · 4 papers
- Tencent · 4 papers
- Technical University of Munich · 3 papers
- Wuhan University · 2 papers
- Peking University · 2 papers
- Xiamen University · 2 papers
- Carnegie Mellon University · 2 papers
- NVIDIA · 2 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Unified LiDAR 3D detection (107 papers)
- Depth-based and label-free 3D detection (58 papers)
- Monocular 3D object detection (56 papers)
Elsewhere in the topic
- LiDAR perception for driving (75 papers)
- Multi-camera BEV detection (69 papers)
- Camera-radar fusion detection (36 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).