Research map / Point clouds / Point cloud classification
Exploring the Potential of Encoder-free Architectures in 3D LMMs
Yiwen Tang, Ziyu Guo, Zhuhao Wang, Renrui Zhang, Qizhi Chen, Junli Liu, Delin Qu, Dong Wang, Bin Zhao, Xuelong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove the 3D encoder with the original token embedding (ablation) | PointLLM · Objaverse classification | GPT-4 score: decrease of 17.5% | Hurt |
| remove the 3D encoder with the original token embedding (ablation) | PointLLM · Objaverse captioning | GPT-4 score: decrease of 10.48% | Hurt |
| add Hybrid Semantic Loss in pre-training (ablation) | encoder-free ENEL-7B · Objaverse captioning | GPT-4 score: 47.19 -> 50.92 | Improved |
| add Hybrid Semantic Loss in pre-training (ablation) | encoder-free ENEL-7B · Objaverse classification | GPT-4 score: 50.61 -> 55.00 | Improved |
| add contrastive loss in pre-training (ablation) | encoder-free 3D LMM · Objaverse classification | GPT-4 score: 47.90 -> 42.50 | Hurt |
| use knowledge distillation loss from Uni3D (ablation) | encoder-free 3D LMM · Objaverse captioning | GPT-4 score: 43.50 -> 43.87 | No clear effect |
| raise mask ratio from 30% to 60% for masked modeling (ablation) | encoder-free 3D LMM · Objaverse captioning | GPT-4 score: 45.20 -> 43.64 | Hurt |
| add masked modeling loss on patches (30% mask) (ablation) | encoder-free 3D LMM · Objaverse classification | GPT-4 score: 47.90 -> 49.00 | Improved |
| add gated self-attention in geometry aggregation (ablation) | ENEL-7B · Objaverse classification | GPT-4 score: 53.50 -> 55.00 | Improved |
| add gated self-attention in geometry aggregation (ablation) | ENEL-7B · Objaverse captioning | GPT-4 score: 49.26 -> 50.92 | Improved |
| increase LLM layers H between aggregation and propagation to 4 or 8 (ablation) | ENEL-7B · Objaverse | GPT-4 score: H=2 best (54.25 classification, 49.56 captioning) | Hurt |
| use 13B Vicuna instead of 7B | ENEL · Objaverse captioning | GPT-4 score: 51.03 -> 53.24 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- PointLLM-7B — PointLLM: Empowering Large Language Models to Understand Point Clouds (ECCV 2024) · better on 6 of 6 shared results
- ShapeLLM-13B — ShapeLLM: Universal 3D Object Understanding for Embodied Interaction (ECCV 2024) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- 3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models (NeurIPS 2026 · similarity 0.90)
- Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning (CVPR 2025 · similarity 0.89)
- More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding (AAAI 2025 · similarity 0.89)
- VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction (CVPR 2026 · similarity 0.88)
- Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding (CVPR 2025 · similarity 0.88)
- LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
