Research map / Vision-language and multimodal / CLIP and zero-shot
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train 3D backbone with ULIP (aligned to frozen CLIP/SLIP image-text space) before fine-tuning | PointMLP · ScanObjectNN | Overall Acc: ↑3.1 | Helped |
| pre-train 3D backbone with ULIP before fine-tuning | PointBERT · ScanObjectNN | Overall Acc: ↑3.3 | Helped |
| pre-train 3D backbone with ULIP before fine-tuning | PointNet++(ssg) · ModelNet40 | Overall Acc: ↑2.7 | Helped |
| pre-train 3D backbone with ULIP before fine-tuning | PointMLP · ModelNet40 | Overall Acc: ↑0.2 | Helped |
| pre-train 3D backbone with ULIP before fine-tuning | PointBERT · ModelNet40 | Overall Acc: ↑0.9 | Helped |
| pre-train PointNeXt with ULIP | PointNeXt · ScanObjectNN | Overall Acc: ↑1.8 | Helped |
| pre-train Group-Free-3D with ULIP | Group-Free-3D · ScanNet-v2 detection | mAP@0.5: ↑1.3 | Helped |
| zero-shot classification with ULIP-aligned 3D encoder instead of PointCLIP projection-based approach | PointBERT · ModelNet40 ALL / Hard | zero-shot top-1 OA: +40.2 on ALL vs PointCLIP; +28.8 on Hard | Helped |
| aligning 3D features to both image and text (three modalities) vs text only (ablation) | PointBERT · ModelNet40 ALL | zero-shot top-1 OA: 60.4 vs 44.7 | Helped |
| aligning 3D features to both image and text vs text only (ablation) | PointBERT · ScanObjectNN ALL | zero-shot top-1 OA: 48.5 vs 31.0 | Helped |
| aligning 3D features to both image and text vs image only (ablation) | PointBERT · ModelNet40 ALL | zero-shot top-1 OA: 60.4 vs 35.5 | Helped |
| ULIP pre-training for data-efficiency (low-data fine-tuning) (ablation) | PointMLP · ScanObjectNN low-data regime | overall accuracy vs training fraction | Helped |
Closest papers by meaning
- ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding (CVPR 2024 · similarity 0.95)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.89)
- Uni3D: Exploring Unified 3D Representation at Scale (ICLR 2024 · similarity 0.89)
- More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding (AAAI 2025 · similarity 0.88)
- 3DCoMPaT200: Language Grounded Large-Scale 3D Vision Dataset for Compositional Recognition (NeurIPS 2024 · similarity 0.88)
- Unifying 2D and 3D Vision-Language Understanding (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.