Research map / Vision-language and multimodal / CLIP and zero-shot
Towards Label-free Scene Understanding by Vision Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SAM refinement of pseudo-labels during training (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 23.7 -> 28.4 | Helped |
| Add SAM refinement of pseudo-labels during training (ablation) | FoundTrans · ScanNet val | 3D mIoU (%): 30.6 -> 33.5 | Helped |
| Apply SAM refinement only to final predictions rather than during training (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 28.1 -> 28.4 | No clear effect |
| Add cross-training between 2D and 3D networks (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 27.7 -> 28.4 | Helped |
| Add cross-training between 2D and 3D networks (ablation) | FoundTrans · ScanNet val | 3D mIoU (%): 31.0 -> 33.5 | Helped |
| Add self-training and cross-training instead of using CLIP labels alone (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 24.7 -> 28.4 | Helped |
| Add self-training and cross-training instead of using CLIP labels alone (ablation) | FoundTrans · ScanNet val | 3D mIoU (%): 30.6 -> 33.5 | Helped |
| Retain CLIP supervision during the second training stage (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 4.7 -> 28.4 | Helped |
| Retain CLIP supervision during the second training stage (ablation) | FoundTrans · ScanNet val | 3D mIoU (%): 5.1 -> 33.5 | Helped |
| Use CLIP, SAM, 2D, and 3D signals for consistency rather than 2D predictions alone (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 6.9 -> 28.4 | Helped |
| Use CLIP, SAM, 2D, and 3D signals for consistency rather than 3D predictions alone (ablation) | FoundTrans · ScanNet val | 3D mIoU (%): 6.4 -> 33.5 | Helped |
| Use SAM features rather than no latent-space regularization (ablation) | FoundTrans · ScanNet val | 2D mIoU (%): 27.1 -> 28.4 | Helped |
Closest papers by meaning
- Hierarchical Intra-modal Correlation Learning for Label-free 3D Semantic Segmentation (CVPR 2024 · similarity 0.89)
- AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction (AAAI 2025 · similarity 0.88)
- Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception (CVPR 2024 · similarity 0.88)
- Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding (NeurIPS 2024 · similarity 0.87)
- A Unified Framework for 3D Scene Understanding (NeurIPS 2024 · similarity 0.87)
- Semi-supervised 3D Semantic Scene Completion with 2D Vision Foundation Model Guidance (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.