Research map / Vision-language and multimodal / CLIP and zero-shot
Blind to Position, Biased in Language: Probing Mid-Layer Representational Bias in Vision-Language Encoders for Zero-Shot Language-Grounded Spatial Understanding
Na Min An, Inha Kang, Minhyun Lee, Hyunjung Shim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add B2G mid-layer P-Map reranking | Global-Local (Mask2Former) · Zero-shot RIS avg, DFN ViT-H/14 | mIoU: 44.64 -> 51.13 | Improved |
| add B2G mid-layer P-Map reranking | HybridGL (SAM) · Zero-shot RIS avg, CLIP ViT-B/16 | mIoU: 45.18 -> 46.24 | Improved |
| add B2G mid-layer P-Map reranking | Global-Local (Mask2Former) · Zero-shot RIS avg, SigLIP ViT-B/16 | mIoU: 46.62 -> 47.70 | Improved |
| choose visual layer dynamically (perm or cluster) instead of fixed mid-layer (ablation) | B2G · Zero-shot RIS, CLIP ViT-B/16 Mask2Former | mIoU: 50.20 vs 49.89 vs 50.13 | No clear effect |
| add B2G spatial guidance | Global-Local+CT+SG · RefCOCOg spatial subset | mIoU: +20.83% mIoU | Improved |
| add B2G spatial guidance | Global-Local+CT+SG · RefCOCOg non-spatial subset | mIoU: +15.55% mIoU | Improved |
| add mid-layer multilingual centroid injection instead of final-layer centroid | SigLIP2 · Multilingual zero-shot RIS (9 benchmarks) | mIoU: 42.42 -> 46.07 -> 49.62 | Improved |
| add mid-layer multilingual centroid injection | SigLIP2 · Multilingual zero-shot RIS (9 benchmarks) | IoU@50: 45.13 -> 48.67 -> 53.41 | Improved |
| add B2G mid-layer centroid | SigLIP2 · Multilingual retrieval, RefCOCO avg | Recall@1: 46.66 -> 50.88 | Improved |
| add B2G mid-layer centroid | SigLIP2 · Multilingual retrieval, RefCOCOg German | Recall@1: 12.85 -> 26.40 | Improved |
| add negation of patch embeddings (ablation) | B2G · RefCOCOg mIoU | mIoU: 49.87 -> 54.92 | Improved |
| add clustering of P-Map (ablation) | B2G · RefCOCOg mIoU | mIoU: 50.36 -> 54.92 | Improved |
Closest papers by meaning
- Grounding Everything: Emerging Localization Properties in Vision-Language Transformers (CVPR 2024 · similarity 0.90)
- Locality Alignment Improves Vision-Language Models (ICLR 2025 · similarity 0.90)
- Mechanisms of Object Localization in Vision-Language Models (CVPR 2026 · similarity 0.90)
- Early Semantic Grounding in Image Editing Models for Zero-Shot Referring Image Segmentation (NeurIPS 2026 · similarity 0.89)
- Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding (CVPR 2025 · similarity 0.89)
- Harnessing Frozen Unimodal Encoders for Flexible Multimodal Alignment (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
