AI Paper Atlas

Research map / Point clouds / Point cloud classification

ShapeLLM: Universal 3D Object Understanding for Embodied Interaction

Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, Kaisheng Ma

ECCV 2024 · Point clouds › Point cloud classification · approach: Point-language and open-vocabulary 3D · 102 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multi-view alignment on top of scaled-up ReCon++ (ablation)ReCon++ · Objaverse-LVIS zero-shotTop-1 accuracy: 51.5 -> 53.7Improved
scale up model and pretraining data of ReCon (ablation)ReCon++ · Objaverse-LVIS zero-shotTop-1 accuracy: 10.3 -> 53.7Improved
scale up model and pretraining data of ReCon (ablation)ShapeLLM · 3D MM-VetTotal: 21.9 -> 53.1Improved
add multi-view alignment on top of scaled-up ReCon++ (ablation)ShapeLLM · 3D MM-VetTotal: 48.2 -> 53.1Improved
replace ULIP-2 encoder in PointLLM-13B with ReCon++ (ablation)PointLLM-13B · 3D MM-VetTotal: +4.20%Improved
train PointLLM on ShapeLLM's GPT-4V instruction dataPointLLM-13B · 3D MM-VetTotalImproved
use 3D point cloud inputs instead of 2D images for 6-DoF groundingShapeLLM-13B · GAPartNet referring groundingAvg accuracy: 6.2 -> 11.7Improved
add absolute position encoding and prompts on top of global representation only (ablation)ShapeLLM-13B · 3D MM-VetTotal: 52.4 -> 53.1Improved
use causal masking instead of random masking in ReCon++ pretraining (ablation)ReCon++-L · ScanObjectNN fine-tune vs zero-shotaccuracy: fine-tune 92.5 -> 95.3; zero-shot 52.8 -> 49.8Mixed
backpropagate global-branch gradients to local branch (no stop-gradient) (ablation)ReCon++-L, random mask · Objaverse-LVIS zero-shotTop-1 accuracy: 52.8 -> 53.7Improved
use ShapeLLM instead of language-only LLMs on 3D MM-VetShapeLLM · 3D MM-VetTotal: 16.2 -> 39.0 (ShapeLLM-7B in language-only table)Improved

Later papers that compared with this method

2 later accepted papers report ShapeLLM-13B as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy