Research map / Point clouds / Point cloud classification
A Unified Approach to Interpreting Self-supervised Pre-training Methods for 3D Point Clouds via Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses game-theoretic interactions to show that self-supervised pre-training on 3D point clouds raises high-order and lowers low-order interaction strength, and proposes an interaction loss that mimics this effect without pre-training.
Problem
The common mechanism behind the effectiveness of different self-supervised pre-training methods for 3D point cloud DNNs is unclear, and pre-training on large unlabeled datasets is costly.
Approach
Decompose a point cloud DNN's output into Harsanyi interactions over point regions and measure normalized average interaction strength per order; pre-training consistently increases high-order and decreases low-order strength. An auxiliary loss that directly enforces this pattern during supervised training achieves comparable gains without pre-training.
Key results
- DGCNN with the proposed interaction loss reaches 93.3% on ModelNet40 and 79.4% on ScanObjectNN, versus 92.5% and 78.1% from scratch, approaching IAE pre-training (94.2%, 85.6%).
- On S3DIS 6-fold, DGCNN with the interaction loss achieves 86.8% OA and 59.0% mIoU, above DGCNN from scratch (84.1%, 56.1%) and above several pre-training methods.
- Zero-shot accuracy on ShapeNet for CrossPoint pre-trained DGCNN is +8.9% over scratch at 1% fine-tuning data but -14.7% at 100%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-training methods (IAE, STRL, CrossPoint, OcCo, Jigsaw) | DGCNN, PointNet, PCN · 3D classification (ModelNet40, ShapeNet, ScanObjectNN) | normalized high-order interaction strength vs low-order | Helped |
| interaction loss term added during training (alpha=0.0005) | PointNet, DGCNN, CurveNet, GDANet · ModelNet40 classification | accuracy (%): DGCNN 92.5→93.3; PointNet 89.2→90.1; CurveNet 92.8→93.1; GDANet 92.3→92.8 | Helped |
| interaction loss term added during training (alpha=0.0005) | PointNet, DGCNN, CurveNet, GDANet · ScanObjectNN PB_T50_RS classification | accuracy (%): DGCNN 78.1→79.4; PointNet 68.0→69.0; CurveNet 79.2→82.0; GDANet 78.7→80.0 | Helped |
| interaction loss term added during training | DGCNN · S3DIS 6-Fold segmentation | OA (%) and mIoU (%): OA 84.1→86.8; mIoU 56.1→59.0 | Mixed |
| interaction loss weight alpha increased from 0.0005 to 0.001 (ablation) | DGCNN · ModelNet40 and ScanObjectNN classification | accuracy (%): ModelNet40 93.3→91.3; ScanObjectNN 79.4→78.1 | Hurt |
| interaction loss weight alpha=0.0001 vs 0 (no loss) (ablation) | DGCNN · ModelNet40 and ScanObjectNN classification | accuracy (%): ModelNet40 92.5→93.0; ScanObjectNN 78.1→79.0 | Helped |
| more pre-training epochs (extent of pre-training) | DGCNN with IAE and CrossPoint · ModelNet40 interaction strength | normalized interaction strength high vs low order | Helped |
| more fine-tuning data (1% to 100%) | DGCNN with IAE and CrossPoint · ModelNet40 interaction strength and accuracy | high-order interaction strength; accuracy gap to scratch: accuracy gap IAE vs scratch narrows to +0.2%; high-order strength gap widens | Mixed |
| CrossPoint pre-training | DGCNN · zero-shot ShapeNet classification (8 common classes) | accuracy (%): +8.9 at 1% fine-tuning data; -14.7 at 100% | Mixed |
Surprising findings
- Pre-trained DNNs can lose transferability: at full fine-tuning data their zero-shot accuracy on ShapeNet falls well below scratch-trained DNNs.
- Excessively large interaction-loss weight (alpha=0.001) reduces accuracy, showing high-order strength has an optimum rather than increasing indefinitely.
Limitations
- Interaction loss is designed for classification and cannot be applied directly to segmentation; it requires a two-stage approach.
- Gains over IAE on ScanObjectNN are substantial in gap (79.4 vs 85.6), so the method does not fully match the best pre-training methods.
- Interaction computation is over only 10 samples per class and n regions, and exact interactions scale as 2^n.
Experimental setup
Datasets: ModelNet40 ShapeNet ScanObjectNN S3DIS
Benchmarks: ModelNet40 ScanObjectNN S3DIS
Metrics: top-1 accuracy overall accuracy mIoU normalized average interaction strength
Baselines: OcCo Jigsaw IAE STRL CrossPoint PointBERT PointMAE PointM2AE PointGPT
Builds on: Harsanyi interaction Shapley interaction index OcCo CrossPoint
Closest papers by meaning
- Self-Supervised Pretraining for Large-Scale Point Clouds (NeurIPS 2022 · similarity 0.89)
- Multi-View Representation is What You Need for Point-Cloud Pre-Training (ICLR 2024 · similarity 0.87)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.87)
- UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting (CVPR 2025 · similarity 0.86)
- Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models (AAAI 2024 · similarity 0.86)
- Point Cloud Pre-training with Diffusion Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.