Research map / Point clouds / Point cloud classification
MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MM-Point, a self-supervised 3D point cloud pre-training method that contrasts DGCNN point cloud features with multiple rendered 2D views (ResNet-50) using Multi-MLP heads and incremental multi-level augmentation.
Problem
Self-supervised representation learning for 3D point clouds is limited by scarce labels; existing 2D-3D contrastive methods do not exploit the complementary information of multiple rendered 2D views of the same object.
Approach
Contrast a 3D point cloud simultaneously with multiple 2D rendered views, using separate projection MLPs per view (Multi-MLP) with dimension varying by view, and apply incrementally stronger augmentations to successive views (multi-level augmentation) to learn multi-level invariance.
Key results
- Linear SVM accuracy of 92.4% on ModelNet40 and 87.8% on ScanObjectNN, versus 91.2% for CrossPoint on ModelNet40 and 81.7% on ScanObjectNN.
- Few-shot 10-way 10-shot on ModelNet40 reaches 90.3% vs CrossPoint 83.6%.
- Semantic segmentation on S3DIS reaches 88.7% OA and 59.1% mIoU, above CrossPoint's 86.7% and 58.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multiple rendered 2D views instead of one (ablation) | MM-Point · ModelNet40 | accuracy: 91.3 -> 92.4 | Helped |
| Use multiple rendered 2D views instead of one (ablation) | MM-Point · ScanObjectNN | accuracy: 83.3 -> 87.8 | Helped |
| Use a Multi-MLP for both intra-modal and inter-modal features (ablation) | MM-Point · ModelNet40 | accuracy: 91.4 -> 92.4 | Helped |
| Use a Multi-MLP for both intra-modal and inter-modal features (ablation) | MM-Point · ScanObjectNN | accuracy: 83.4 -> 87.8 | Helped |
| Add multi-view augmentation (ablation) | MM-Point · ModelNet40 | accuracy: 91.7 -> 92.1 | Helped |
| Add multi-view augmentation (ablation) | MM-Point · ScanObjectNN | accuracy: 86.1 -> 86.9 | Helped |
| Use hierarchical multi-level augmentation rather than non-hierarchical multi-view augmentation (ablation) | MM-Point · ModelNet40 | accuracy: 92.1 -> 92.4 | Helped |
| Use hierarchical multi-level augmentation rather than non-hierarchical multi-view augmentation (ablation) | MM-Point · ScanObjectNN | accuracy: 86.9 -> 87.8 | Helped |
| Pretrain with MM-Point rather than random initialization | DGCNN · S3DIS Area 5 validation | mIoU: +4.2% mIoU | Helped |
Limitations
- Ablation of number of views shows gains saturate beyond 4 views, and the optimal view count is chosen on evaluation results.
- Relies on rendering multiple 2D views per object, adding preprocessing cost.
- Evaluation largely uses linear SVM on frozen features with a small number of baselines.
Experimental setup
Datasets: ShapeNet ModelNet40 ModelNet10 ScanObjectNN ShapeNetPart S3DIS
Benchmarks: ModelNet40 ModelNet10 ScanObjectNN ShapeNetPart S3DIS
Metrics: accuracy few-shot accuracy overall accuracy mIoU
Baselines: CrossPoint Point-MAE Point-BERT OcCo Jigsaw3D STRL SwAV MaskPoint
Builds on: CrossPoint CMC InfoMin InfoMax
Closest papers by meaning
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.88)
- Partial Point Cloud Registration with Multi-view 2D Image Learning (AAAI 2025 · similarity 0.87)
- MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations (NeurIPS 2024 · similarity 0.87)
- MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding (AAAI 2025 · similarity 0.87)
- Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning (CVPR 2025 · similarity 0.87)
- Cross-Modal 3D Representation with Multi-View Images and Point Clouds (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.