Research map / Point clouds / Point cloud classification
General Point Model Pretraining with Autoencoding and Autoregressive
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes General Point Model (GPM), a point cloud transformer that jointly trains masked point prediction (autoencoding) and autoregressive token generation on dVAE-tokenized ShapeNet patches, evaluated on classification, segmentation, few-shot, and unconditional generation.
Problem
Point cloud pretraining with transformers has mostly used autoencoding (masked prediction) alone, which lacks generation ability and does not model dependencies among masked regions; no single transformer handles both point cloud understanding and generation.
Approach
Discretize point cloud patches with a dVAE tokenizer, then concatenate a masked autoencoding segment (Part A) and an autoregressive segment (Part B) in one transformer with a special attention mask, combining MPM, autoregressive token prediction, and contrastive learning with point patch mixing.
Key results
- GPM reaches 94.0% on ModelNet40 with 1k points and 94.3% with 8k points, outperforming Point-BERT (93.2% and 93.8%).
- On ScanObjectNN, GPM improves over Point-BERT by about 2.77, 1.88 and 1.73 points across the three variants.
- Unconditional generation on ShapeNet shows GPM with lower JSD than Point-Flow and Point-BERT on airplane, chair and car categories.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add autoregressive mask generation to autoencoding (ablation) | GPM · ModelNet40 | accuracy: 92.9 -> 93.8 | Helped |
| reverse the order of autoregressive and autoencoding tasks (ablation) | GPM · ModelNet40 | accuracy: 93.8 -> 93.6 | No clear effect |
| add autoregressive mask generation to autoencoding (ablation) | GPM · ModelNet40 | accuracy: 93.4 -> 93.9 | Helped |
| reverse the order of autoregressive and autoencoding tasks (ablation) | GPM · ModelNet40 | accuracy: 93.9 -> 93.9 | No clear effect |
Surprising findings
- Order of autoencoding and autoregressive tasks has little effect on understanding accuracy (93.8 vs 93.6 at 1k points).
Limitations
- Gains over Point-MAE and Point-M2AE on several benchmarks are small or absent (e.g., ScanObjectNN PB-T50-RS 84.8 vs Point-MAE 85.2).
- Conditional generation is only shown qualitatively or in supplementary material, not quantitatively in the main text.
- Generation evaluation is limited to ShapeNet airplane, chair and car categories.
Experimental setup
Datasets: ShapeNet ModelNet40 ScanObjectNN ShapeNetPart
Benchmarks: ModelNet40 ScanObjectNN ShapeNetPart ShapeNet generation
Metrics: accuracy mIoU few-shot accuracy JSD MMD Coverage 1-NNA
Baselines: Point-BERT MaskPoint Point-MAE Point-M2AE Point-Flow PointGPT OcCo ACT
Builds on: Point-BERT GLM Point-MAE MoCo DALL-E dVAE
Closest papers by meaning
- LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling (NeurIPS 2024 · similarity 0.87)
- PointMamba: A Simple State Space Model for Point Cloud Analysis (NeurIPS 2024 · similarity 0.86)
- PointDGMamba: Domain Generalization of Point Cloud Classification via Generalized State Space Model (AAAI 2025 · similarity 0.86)
- 3D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining (ICLR 2024 · similarity 0.85)
- PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learning (AAAI 2025 · similarity 0.85)
- Adaptive Siamese Masked Autoencoder with Global Optimization for Unsupervised Point Cloud Shape Correspondence (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.