Research map / Vision-language and multimodal / CLIP and zero-shot
Reconstructing CLIP for Open-Vocabulary Dense Perception
Yajie Liu, Jinjin Zhang, Qingjie Liu, Di Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-layer value embeddings instead of final spatial features as dense basis (ablation) | DenseRC · OV-COCO | AP50 novel: 41.3 -> 44.4 | Improved |
| use multi-layer value embeddings instead of single last-layer value (ablation) | DenseRC · OV-COCO | AP50 novel: 41.9 -> 44.4 | Improved |
| add vanilla spatial self-attention to dense feature construction (ablation) | DenseRC · OV-COCO | AP50 novel: 44.4 -> 39.1 | Hurt |
| add self-self (Q-Q) attention to dense feature construction (ablation) | DenseRC · OV-COCO | AP50 novel: 44.4 -> 42.3 | Hurt |
| add Head-Selective Gating (ablation) | DenseRC · OV-COCO | AP50 novel: 44.4 -> 45.6 | Improved |
| add Head-Selective Gating (ablation) | DenseRC · ADE20K-847 | mIoU: 15.7 -> 15.9 | Improved |
| aggregate last three layers' value embeddings instead of last layer only (ablation) | DenseRC · OV-COCO | AP50 novel: 42.3 -> 45.6 | Improved |
| aggregate four layers instead of three (ablation) | DenseRC · OV-COCO | AP50 novel: 45.6 -> 43.8 | Hurt |
| increase feature coherence loss weight lambda from 0.025 to 0.1 (ablation) | DenseRC · ADE20K-150 | mIoU: 37.6 -> 36.8 | Hurt |
| change lambda from 0.025 to 0.04 (ablation) | DenseRC · PC-59 | mIoU: 61.3 -> 61.4 | No clear effect |
| use DenseRC instead of DeCLIP for detector backbone | F-ViT · OV-COCO | AP50 novel: 41.1 -> 45.6 | Improved |
| use DenseRC instead of DeCLIP in CAT-Seg | CAT-Seg · ADE20K-150 | mIoU: 36.3 -> 37.6 | Improved |
Closest papers by meaning
- DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception (CVPR 2025 · similarity 0.90)
- CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction (ICLR 2024 · similarity 0.90)
- Unveiling the Knowledge of CLIP for Training-Free Open-Vocabulary Semantic Segmentation (AAAI 2025 · similarity 0.89)
- ResCLIP: Residual Attention for Training-free Dense Vision-language Inference (CVPR 2025 · similarity 0.89)
- Unlocking Fine-Grained Perception in CLIP via Structurally-Aware Latent Masked Modeling (NeurIPS 2026 · similarity 0.89)
- Refining CLIP's Spatial Awareness: A Visual-Centric Perspective (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
