Research map / Point clouds / Point cloud classification
Point-language and open-vocabulary 3D
Align 3D point clouds with language for zero-shot and open-vocabulary understanding.
107 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Point cloud classification, keyword cluster “language · pre · understanding”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 2 of 3 tracked benchmarks.
35 papers in 2023–24 and 59 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 15.1% to 22.8%.
Open questions the data points to
- ModelNet40 (accuracy): best went from 94.08 (up to 2024) to 96.23 in 2025 — MoST: Efficient Monarch Sparse Tuning for 3D Representation Learning (CVPR 2025 · 0 citations)
- S3DIS Area 5 (mIoU): best went from 72.7 (up to 2024) to 75.1 in 2025 — On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation (NeurIPS 2025 · 1 citation)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (CVPR 2020 · 135 citations)
- 2021 · Scan2Cap: Context-Aware Dense Captioning in RGB-D Scans (CVPR 2021 · 136 citations)
- 2022 · PointCLIP: Point Cloud Understanding by CLIP (CVPR 2022 · 435 citations)
- 2023 · MVImgNet: A Large-Scale Dataset of Multi-View Images (CVPR 2023 · 104 citations)
- 2024 · LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning (CVPR 2024 · 84 citations)
- 2025 · Lift3D Policy: Lifting 2D Foundation Models for Robust 3D Robotic Manipulation (CVPR 2025 · 9 citations)
- 2026 · HCC-3D: Hierarchical Compensatory Compression for 98% 3D Token Reduction in Vision-Language Models (AAAI 2026 · 1 citation)
Earliest papers here
- Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (CVPR 2020 · 135 citations)
- Scan2Cap: Context-Aware Dense Captioning in RGB-D Scans (CVPR 2021 · 136 citations)
- Learning Geometry-Disentangled Representation for Complementary Understanding of 3D Object Point Cloud (AAAI 2021 · 129 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ModelNet40 · accuracy (higher is better)
6 papers, 2020–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 93.02 | Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (CVPR 2020 · 135 citations) |
| 2021 | 93.8 | Learning Geometry-Disentangled Representation for Complementary Understanding of 3D Object Point Cloud (AAAI 2021 · 129 citations) |
| 2022 | 94.08 | PointCLIP: Point Cloud Understanding by CLIP (CVPR 2022 · 435 citations) |
| 2025 | 96.23 | MoST: Efficient Monarch Sparse Tuning for 3D Representation Learning (CVPR 2025 · 0 citations) |
S3DIS Area 5 · mIoU (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 72.7 | Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training (CVPR 2024 · 51 citations) |
| 2025 | 75.1 | On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation (NeurIPS 2025 · 1 citation) |
ScanObjectNN · accuracy (higher is better)
4 papers, 2020–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 87.2 | Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (CVPR 2020 · 135 citations) |
| 2024 | 87.8 | MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding (AAAI 2024 · 14 citations) |
What has been tried
73 papers here report 843 measured changes (a component added, removed or swapped, compared with a baseline): helped 77%, no clear effect 3%, hurt 18%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Replace direct visual-prompt injection with early fusion in the Multi-Modal Transformer | ScanRefer · C@0.5 | 59.39 -> 62.84; +3.45% | Helped | LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning (CVPR 2024 · 84 citations) |
| use fused 2D and 3D features | 3DSSG · object R@5 | 0.46 -> 0.57 | Helped | Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships (CVPR 2024 · 54 citations) |
| add Prompt-driven Normalization instead of no prompt adapter | ScanNet 20-category semantic segmentation, joint training · mIoU | 68.9 -> 75.7 | Helped | Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training (CVPR 2024 · 51 citations) |
| add geometry-aware point-cloud sampling instead of random sampling | Anomaly-ShapeNet · I-AUC | 0.55 -> 0.66 | Helped | Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network (CVPR 2024 · 51 citations) |
| add Level 1 annotations to Level 2 training data | ObjaverseXL-LVIS QA test · Acc | 22.1 -> 25.4 | Helped | GPT4Point: A Unified Framework for Point-Language Understanding and Generation (CVPR 2024 · 34 citations) |
| Add hybrid sampling rather than using top-1 token probabilities | DeepCAD test · mean CD (×10−3) | 6.784 -> 3.432 | Helped | CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention (CVPR 2024 · 28 citations) |
| add intra-mask smoothing loss to inter-mask contrastive loss | ScanNet 10 classes · mIoU | 35.24 -> 38.29 | Helped | OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding (NeurIPS 2024 · 27 citations) |
| Add word-level grammar parsing rather than using sentence-level features | 3DSSG PREDCLS · mR@20 | 6.8 -> 29.4 | Helped | CLIP-Driven Open-Vocabulary 3D Scene Graph Generation via Cross-Modality Contrastive Learning (CVPR 2024 · 24 citations) |
| add both Point-prior Prompt and Geometry-aware Adapter to the baseline | ScanObjectNN PB-T50-RS · accuracy | +0.5% | Helped | Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models (AAAI 2024 · 21 citations) |
| Add edge features to graph encoding | 3DSSG test · Node R@1 | 72.53 -> 75.33 | Helped | SGFormer: Semantic Graph Transformer for Point Cloud-Based 3D Scene Graph Generation (AAAI 2024 · 21 citations) |
| add LLM-estimated thickness rather than integrate occupied voxels from depth-based carving | ABO-500 val · ADE | 18.587 -> 9.786 | Helped | Physical Property Understanding from Language-Embedded Feature Fields (CVPR 2024 · 15 citations) |
| Use multiple rendered 2D views instead of one | ModelNet40 · accuracy | 91.3 -> 92.4 | Helped | MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding (AAAI 2024 · 14 citations) |
What reviewers will expect
Datasets and baselines named in the 35 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ScanNet · 6 of 35 papers
- ModelNet40 · 6 of 35 papers
- ShapeNet · 5 of 35 papers
- ScanRefer · 4 of 35 papers
- Objaverse · 4 of 35 papers
- ScanObjectNN · 4 of 35 papers
- Objaverse-LVIS · 3 of 35 papers
- PartNet-Mobility · 2 of 35 papers
Baselines
- OpenShape · 4 of 35 papers
- 3D-VisTA · 3 of 35 papers
- ReCon · 3 of 35 papers
- Uni3D · 3 of 35 papers
- ReLA · 2 of 35 papers
- InstanceRefer · 2 of 35 papers
- MCLN · 2 of 35 papers
- EDA · 2 of 35 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 94 papers; from OpenAlex and the papers' affiliations).
- The Chinese University of Hong Kong · 7 papers
- Peking University · 6 papers
- Shanghai AI Laboratory · 5 papers
- Sun Yat-sen University · 5 papers
- National University of Singapore · 4 papers
- Tsinghua University · 4 papers
- Fudan University · 4 papers
- Shanghai Jiao Tong University · 4 papers
- Hong Kong University of Science and Technology · 4 papers
- The University of Hong Kong · 4 papers
Use it
Download BibTeX of 36 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Self-supervised point cloud pretraining (73 papers)
- Masked point autoencoders (22 papers)
Elsewhere in the topic
- Local feature aggregation networks (158 papers)
- Scene flow and dynamic point clouds (89 papers)
- Equivariant point cloud networks (84 papers)
- LiDAR point cloud compression and completion (60 papers)
- Point cloud detection and tracking (56 papers)
- Efficient point sampling architectures (45 papers)
- Adversarial point cloud attacks (37 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).