Research map / Point clouds / Point cloud classification
3D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MaskFeat3D, a point cloud masked autoencoder that predicts surface normals and surface variations at masked points (instead of positions) using an attention-based decoder that is independent of the encoder.
Problem
Existing 3D masked autoencoders for point clouds reconstruct masked point locations, which may underuse intrinsic surface features needed for 3D analysis.
Approach
Replace position reconstruction with prediction of point normals and surface variations at masked points, using a cross-attention plus self-attention decoder that takes masked points as queries and encoder block features as keys/values, so it works with ViT, PointNet++-style, and sparse CNN encoders.
Key results
- On ScanObjectNN PB-T50-RS, MaskFeat3D (PointViT) reaches 87.7% vs. 85.7% for MaskSurfel and 85.2% for PointMAE.
- MaskFeat3D with PointNeXt encoder reaches 92.7% OBJ-BG and 88.6% PB-T50-RS on ScanObjectNN, and 87.4% ins. mIoU on ShapeNetPart, state of the art among the compared methods.
- On ScanNet object detection with CAGroup3D, MaskFeat3D improves AP25 to 75.6 and AP50 to 62.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace masked-point position reconstruction with feature prediction (normals and surface variation) | PointViT-based pretraining · ScanObjectNN PB-T50-RS | accuracy: 85.7% to 87.7% | Helped |
| Use normals and surface variation together instead of either feature alone (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 86.5, 87.0, 87.7 | Helped |
| Add self-attention layers to the decoder (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 85.7 to 87.7; -2.0 | Helped |
| Increase decoder depth from 2 to 4 blocks (ablation) | MaskFeat3D · ScanObjectNN | accuracy: +1.5% | Helped |
| Increase decoder depth beyond 4 blocks (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 87.7 to 87.5 to 87.1 | Hurt |
| Set mask ratio to 60% rather than 40% or 90% (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 86.8, 87.7, 86.5 | Helped |
| Use all masked points as decoder queries rather than a subset (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 85.7, 86.2, 86.6, 87.7 | Helped |
| Use rotation and scaling augmentation together (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 87.0, 85.9, 87.7 | Helped |
| Add translation to rotation and scaling augmentation (ablation) | MaskFeat3D · ScanObjectNN | accuracy: 87.7 to 86.7 | Hurt |
| Replace PointViT with PointNeXt as the encoder | MaskFeat3D · ScanObjectNN OBJ-BG | accuracy: 91.9 to 92.7 | Helped |
| Use surface variation and color as scene-pretraining target features | MaskFeat3D with VoteNet backbone · ScanNet object detection | AP50: 38.4 to 41.0 | Helped |
Surprising findings
- Predicting normals and surface variations at masked points with position-index matching for PointMAE-style decoders gives only modest gains, far below the proposed decoder, suggesting decoder design matters as much as the target.
- A 1-block decoder already reaches 85.8% on ScanObjectNN, above the runner-up PointMAE.
Limitations
- Scene-level extension is preliminary and not the main focus; uses only surface variation and color there.
- Query-point subsampling is needed to limit quadratic self-attention cost on large masked sets.
Experimental setup
Datasets: ShapeNet ModelNet40 ScanObjectNN ShapeNetPart ScanNet SUN RGB-D
Benchmarks: ScanObjectNN ShapeNetPart ModelNet40 ScanNet SUN RGB-D
Metrics: accuracy instance mIoU class mIoU AP25 AP50
Baselines: PointViT PointBERT MaskDiscr MaskSurfel PointMAE OcCo CrossPoint PointNeXt MinkowskiNet Point-M2AE VoteNet CAGroup3D
Builds on: Point-MAE PointBERT masked discrimination PointNeXt
Closest papers by meaning
- Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders (AAAI 2024 · similarity 0.91)
- RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learning (AAAI 2025 · similarity 0.90)
- PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders (NeurIPS 2024 · similarity 0.90)
- Adaptive Siamese Masked Autoencoder with Global Optimization for Unsupervised Point Cloud Shape Correspondence (AAAI 2025 · similarity 0.88)
- Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds (CVPR 2025 · similarity 0.88)
- BEV-MAE: Bird’s Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.