Research map / Point clouds / Point cloud classification
More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GreenPLM, a 3-stage text-heavy training strategy (6M generated text descriptions/conversations, then 90K 3D pairs) with a zero-parameter cross-attention token pooling (0M-Pooling) to make point-language LLMs data-efficient.
Problem
Point-language LLMs require large amounts of scarce 3D-text pair data; the paper asks whether robust 3D object understanding is possible with minimal 3D data.
Approach
Use the frozen pre-trained point-text encoder (Uni3D) so text-only data can train the text-to-LLM alignment, shifting the data bottleneck from 3D-text pairs to abundant text; add a parameter-free cross-attention pooling of point tokens and a new 6M-sample T3D text dataset.
Key results
- GreenPLM-0 (text-only) reaches 54.57% average classification accuracy with no 3D data, above LLaVA-1.5-13B (32.62%) and Point-Bind LLM (26.68%).
- GreenPLM with 90K 3D pairs achieves 60.08% average accuracy, +10.95 over PointLLM-7B (90K) and +11.06 over MiniGPT-3D (90K), and comparable to MiniGPT-3D trained on 730K.
- On captioning, GreenPLM (90K) scores 2.15 Qwen2 points above PointLLM-13B trained on 730K 3D data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train GreenPLM using only text data, with no 3D training pairs | GreenPLM-0 · ModelNet40 test and Objaverse | Average accuracy: 26.68 -> 54.57 | Helped |
| Use GreenPLM with 90K 3D training pairs instead of PointLLM-7B with 90K | GreenPLM · ModelNet40 test and Objaverse | Average accuracy: 49.13 -> 60.08 | Helped |
| Use GreenPLM with 90K 3D training pairs instead of MiniGPT-3D with 90K | GreenPLM · Objaverse | Qwen2 caption score: 35.05 -> 42.55 | Helped |
| Use GreenPLM with 90K 3D training pairs instead of MiniGPT-3D with 90K | GreenPLM · Objaverse | Sentence-BERT caption similarity: 46.68 -> 49.40 | Helped |
| Use GreenPLM with 90K 3D training pairs instead of MiniGPT-3D with 90K | GreenPLM · Objaverse | SimCSE caption similarity: 47.75 -> 49.36 | Helped |
| Add Stage III point-cloud training after Stages I and II (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy: 54.57 -> 60.08 | Helped |
| Add Stage II after Stages I and III (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy: 58.25 -> 60.08 | Helped |
| Add Stage I before Stages II and III (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy: 42.78 -> 60.08 | Helped |
| Add pooled point tokens to class and global tokens (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy: 45.42 -> 60.08 | Helped |
| Replace 0M-Pooling with Max Pooling (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy: drops by 1.96 | Hurt |
| Replace 0M-Pooling with Mean Pooling (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy: drops by 1.58 | Hurt |
| Replace T3D captions with low-information daily conversation data in Stage I (ablation) | GreenPLM · ModelNet40 and Objaverse classification | Accuracy | Hurt |
Surprising findings
- Text-only trained GreenPLM-0 surpasses GPT4Point and performs on par with PointLLM-7B trained on 730K 3D samples on classification.
- Using daily conversation text instead of captions as text-encoder inputs causes a significant performance drop, indicating caption content matters.
Limitations
- Exploration of text and 3D data combinations was limited by time and resources.
- Only tested on small single objects, not larger scenes.
- Evaluation replaces GPT-based judging with Qwen2-72B-Instruct, a new judge whose agreement with prior metrics is not validated in the text.
Experimental setup
Datasets: T3D (6M generated text) Cap3D Point-text instruction dataset (PointLLM, 90K) ModelNet40 Objaverse
Benchmarks: ModelNet40 Objaverse
Metrics: accuracy A3DR Qwen2 score Sentence-BERT similarity SimCSE similarity
Baselines: InstructBLIP LLaVA-1.5 GPT-4o mini Point-Bind LLM PointLLM MiniGPT-3D GPT4Point
Builds on: PointLLM Uni3D MiniGPT-3D LoRA Cap3D
Closest papers by meaning
- 3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models (NeurIPS 2026 · similarity 0.92)
- GPT4Point: A Unified Framework for Point-Language Understanding and Generation (CVPR 2024 · similarity 0.90)
- PerLA: Perceptive 3D Language Assistant (CVPR 2025 · similarity 0.90)
- LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding (AAAI 2025 · similarity 0.90)
- Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding (CVPR 2025 · similarity 0.89)
- Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.