Research map / Vision-language and multimodal / CLIP and zero-shot
Zero-Shot Depth Completion with Vision-Language Model
Zhiqiang Yan, Yuan Wu, Gim Hee Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use zero-initialized fusion convolutions instead of He-initialized (ablation) | SDIM visual tokenization · VOID 150 | RMSE (m): 0.630 -> 0.623 | Improved |
| use fusion block with He init instead of direct RGB-D concatenation (ablation) | SDIM visual tokenization · VOID 150 | RMSE (m): 0.635 -> 0.630 | Improved |
| add binary mask as textual prompt (ablation) | SDIM · VOID 150 | RMSE (m): 0.623 -> 0.608 | Improved |
| add sparse depth values to the textual prompt (ablation) | SDIM · VOID 150 | RMSE (m): 0.608 -> 0.604 | Improved |
| supervise with dense ground-truth depth instead of sparse depth text (ablation) | SDIM · VOID 150 | RMSE (m): 0.604 -> 0.592 | Improved |
| use Qwen2.5-VL backbone instead of Seed1.5-VL (ablation) | SDIM · IBims-1 | MAE (m): 0.048 -> 0.040 | Improved |
| use Qwen2.5-VL backbone instead of MolmoE-1B (ablation) | SDIM · IBims-1 | MAE (m): 0.054 -> 0.040 | Improved |
| use VLM-based completion instead of Marigold-DC diffusion | Marigold-DC · IBims-1 | inference speed: 0.005 -> 0.327 FPS | Improved |
| use VLM-based completion instead of task-specific models | CFormer / BPNet · IBims-1 | memory (GB): 0.69 -> 46.10 | Hurt |
| use VLM-based completion on DDAD versus VPP4DC | Ours (w/ SD) · DDAD | MAE (m): 1.344 -> 1.353 | Hurt |
Closest papers by meaning
- DepthLM: Metric Depth from Vision Language Models (ICLR 2026 · similarity 0.92)
- VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion (AAAI 2025 · similarity 0.90)
- On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation (CVPR 2024 · similarity 0.89)
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.89)
- Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.89)
- Unifying 2D and 3D Vision-Language Understanding (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
