Research map / Point clouds / Segmentation and detection
CLIP2Scene: Towards Label-Efficient 3D Scene Understanding by CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| transfer CLIP knowledge with semantic and spatial-temporal consistency regularization | 3D point-cloud segmentation network · nuScenes val, 1% labels | mIoU: 48.2 -> 56.3 | Helped |
| transfer CLIP knowledge with semantic and spatial-temporal consistency regularization | 3D point-cloud segmentation network · nuScenes val, 100% labels | mIoU: 70.4 -> 71.5 | Helped |
| use class names from SemanticKITTI rather than nuScenes in prompts (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 15.1 -> 13.9 (-1.2) | Hurt |
| use class names from Cityscapes rather than nuScenes in prompts (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 15.1 -> 11.3 (-3.8) | Hurt |
| include class names from all datasets in prompts (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 15.1 -> 15.3 (+0.2) | Helped |
| add Spatial-temporal Consistency Regularization (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 19.8 -> 20.8 | Helped |
| add Semantic Consistency Regularization (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 16.8 -> 20.8 | Helped |
| replace both regularizers with KL divergence distillation (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 15.1 -> 0.0 (-15.1) | Hurt |
| add Switchable Self-Training Strategy (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 18.8 -> 20.8 | Helped |
| train only on the 3D network's own predictions after ten epochs (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 15.1 -> 10.1 (-4.0) | Hurt |
| increase the input from 1 sweep to 3 sweeps (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 18.7 -> 20.8 | Helped |
| increase the input from 3 sweeps to 5 sweeps (ablation) | CLIP2Scene · nuScenes val, annotation-free | mIoU: 20.8 -> 20.6 | No clear effect |
Closest papers by meaning
- CLIP-Driven Open-Vocabulary 3D Scene Graph Generation via Cross-Modality Contrastive Learning (CVPR 2024 · similarity 0.89)
- Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception (CVPR 2024 · similarity 0.89)
- Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding (AAAI 2024 · similarity 0.89)
- RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding (CVPR 2024 · similarity 0.88)
- A Unified Framework for 3D Scene Understanding (NeurIPS 2024 · similarity 0.88)
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.