Research map / Point clouds / Point cloud classification
PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PCP-MAE, a point masked autoencoder that learns to predict masked patch centers via a weight-shared cross-attention Predicting Center Module and feeds the predicted (stop-gradient) centers to the decoder instead of ground-truth centers.
Problem
Point-MAE-style masked autoencoders feed ground-truth masked patch centers to the decoder, which leaks position information so the reconstruction task is trivial and the encoder may not learn semantic features.
Approach
Show that a decoder given only masked centers can reconstruct point clouds at 100% masking, then add a Predicting Center Module that shares Transformer weights with the encoder and uses cross-attention to predict masked centers' positional embeddings (with an L2 loss), replacing ground-truth centers in the decoder input with stop-gradient predicted embeddings.
Key results
- PCP-MAE reaches 95.52% on ScanObjectNN OBJ-BG, 94.32% on OBJ-ONLY and 90.35% on PB-T50-RS, improving Point-MAE by 5.50, 6.03 and 5.17 points.
- On ModelNet40 few-shot, PCP-MAE gains 1.1, 1.3, 0.9, 0.9 points over Point-MAE across the four 5-way/10-way settings.
- Predicting sin-cos positional embeddings outperforms predicting raw coordinates (90.35 vs 88.68 on PB-T50-RS); removing stop-gradient drops PB-T50-RS from 90.35 to 87.36.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add center prediction with PCM to reconstruction pre-training (ablation) | PCP-MAE · ScanObjectNN OBJ-BG | accuracy: 92.94 -> 94.32 | Helped |
| replace decoder ground-truth masked centers with predicted centers (ablation) | PCP-MAE with reconstruction and PCM · ScanObjectNN OBJ-BG | accuracy: 94.32 -> 95.52 | Helped |
| add stop-gradient to predicted centers before the decoder (ablation) | PCP-MAE · ScanObjectNN OBJ-BG | accuracy: 92.42 -> 95.52 | Helped |
| predict sin-cos positional embeddings instead of 3D coordinates (ablation) | PCP-MAE center prediction · ScanObjectNN OBJ-BG | accuracy: 94.32 -> 95.52 | Helped |
| share encoder and PCM parameters (ablation) | PCP-MAE · ScanObjectNN OBJ-BG | accuracy: 94.66 -> 95.52 | Helped |
| add one transformer layer to the center-prediction projector (ablation) | PCP-MAE · ScanObjectNN PB-T50-RS | accuracy: 90.35 -> 88.93 | Hurt |
| use l2 Distance rather than l1 Distance for center prediction (ablation) | PCP-MAE · ScanObjectNN OBJ-BG | accuracy: 94.49 -> 95.52 | Helped |
| use Scale&Translate + Rotation pre-training augmentation instead of no augmentation (ablation) | PCP-MAE · ScanObjectNN OBJ-BG | accuracy: 94.19 -> 95.52 | Helped |
| use Scale&Translate + Rotation rather than Rotation + Scale&Translate (ablation) | PCP-MAE pre-training · ScanObjectNN PB-T50-RS | accuracy: 90.18 -> 90.35 | Helped |
| use PCP-MAE instead of Point-MAE | point-cloud pre-training and fine-tuning · ScanObjectNN OBJ-BG | accuracy: +5.50 | Helped |
| use PCP-MAE instead of Point-MAE | point-cloud pre-training and fine-tuning · ScanObjectNN OBJ-ONLY | accuracy: +6.03 | Helped |
| use PCP-MAE instead of Point-MAE | point-cloud pre-training and fine-tuning · ScanObjectNN PB-T50-RS | accuracy: +5.17 | Helped |
Surprising findings
- Point-MAE decoder reconstructs point clouds well even with a 100% masking ratio when given only masked center positional embeddings, unlike 2-D MAE.
- Applying the proposed PCP-MAE augmentation to prior SOTA methods (Point-FEMAE, ReCon) slightly degrades their performance.
Limitations
- Single-modal method; does not exploit cross-modal or contrastive learning benefits.
- Pre-training data (point clouds) is limited in size.
- Pre-training takes slightly longer than Point-MAE and parameters grow relative to Point-MAE due to the PCM and projector.
Experimental setup
Datasets: ShapeNet ScanObjectNN ModelNet40 ShapeNetPart S3DIS
Benchmarks: ScanObjectNN ModelNet40 ShapeNetPart S3DIS
Metrics: accuracy mIoU mAcc Cls.mIoU Inst.mIoU
Baselines: Point-MAE Point-M2AE Point-FEMAE ACT I2P-MAE ReCon PointGPT Point-BERT
Builds on: Point-MAE MAE Point-BERT ReCon
Closest papers by meaning
- 3D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining (ICLR 2024 · similarity 0.90)
- RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learning (AAAI 2025 · similarity 0.89)
- Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders (AAAI 2024 · similarity 0.89)
- Self-Guided Masked Autoencoder (NeurIPS 2024 · similarity 0.87)
- R-MAE: Regions Meet Masked Autoencoders (ICLR 2024 · similarity 0.86)
- LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.