Research map / Point clouds / Point cloud classification
PointLLM: Empowering Large Language Models to Understand Point Clouds
Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, Dahua Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use 3 projection layers instead of none (ablation) | PointLLM-7B · Generative 3D classification avg | accuracy: 50.63 -> 52.82 | Improved |
| use 4 projection layers instead of 3 (ablation) | PointLLM-13B · Generative 3D classification avg | accuracy: 53.39 -> 51.40 | Hurt |
| add max pooling of point tokens before projection (ablation) | PointLLM-7B · Generative 3D classification avg | accuracy: 52.82 -> 48.72 | Hurt |
| add max pooling of point tokens before projection (ablation) | PointLLM-7B · training cost | A100 GPU-hours: 126 -> 34 | Improved |
| add multi-round conversation data to fine-tuning (ablation) | PointLLM-7B · Generative 3D classification avg | accuracy: 40.14 -> 45.79 | Improved |
| add detailed description data to fine-tuning (ablation) | PointLLM-7B · Generative 3D classification avg | accuracy: 45.79 -> 52.82 | Improved |
| increase alignment data volume | PointLLM-7B · Generative 3D classification | accuracy | Improved |
| use point cloud input instead of single-view images | PointLLM · Generative 3D classification avg | accuracy: 44.86 -> 52.82 (LLaVA-7B vs PointLLM-7B) | Improved |
| use 13B backbone instead of 7B | PointLLM · Objaverse 3D captioning | hallucination: 0.66 -> 0.84 | Hurt |
| prompt for captions of at most 20 words (ablation) | PointLLM-13B · Objaverse 3D captioning | BLEU-1: 3.83 -> 17.09 BLEU-1; correctness 3.10 -> 2.12 | Mixed |
Later papers that compared with this method
7 later accepted papers report PointLLM-7B as a baseline in their results tables; 7 of them report a better number on most of the results they share with it.
- Exploring the Potential of Encoder-free Architectures in 3D LMMs (ICLR 2026 · better on 6 of 6 shared results)
- PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models (CVPR 2026 · better on 5 of 5 shared results)
- HCC-3D: Hierarchical Compensatory Compression for 98% 3D Token Reduction in Vision-Language Models (AAAI 2026 · better on 2 of 2 shared results)
- 4DPC$^2$hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping (ICML 2026 · better on 1 of 1 shared result)
- Test-Time Optimization of 3D Point Cloud LLM via Manifold-Aware In-Context Guidance and Refinement (ICLR 2026 · better on 1 of 1 shared result)
- More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding (AAAI 2025 · better on 1 of 1 shared result)
- ShapeLLM: Universal 3D Object Understanding for Embodied Interaction (ECCV 2024 · better on 3 of 3 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding (AAAI 2025 · similarity 0.92)
- 3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models (NeurIPS 2026 · similarity 0.91)
- GPT4Point: A Unified Framework for Point-Language Understanding and Generation (CVPR 2024 · similarity 0.90)
- PerLA: Perceptive 3D Language Assistant (CVPR 2025 · similarity 0.89)
- LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding (AAAI 2025 · similarity 0.88)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
