Research map / Vision-language and multimodal / CLIP and zero-shot
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CLIP visual prompts and fine-tune LayerNorm for category-level retrieval | CLIP-based ZS-SBIR · Sketchy (extended), TU-Berlin, QuickDraw Extended | mAP: 0.723 vs 0.702; 0.651 vs 0.637; 0.202 vs 0.188 | Helped |
| Use CLIP visual prompts and fine-tune LayerNorm for cross-category fine-grained retrieval | CLIP-based FG-ZS-SBIR · Sketchy | Top-1: 28.68 vs 27.62 | Helped |
| Fine-tune LayerNorm parameters (ablation) | CLIP-based ZS-SBIR and FG-ZS-SBIR · Sketchy | Top-1: 27.18 -> 28.68 | Helped |
| Use the CLIP text-encoder classification loss (ablation) | CLIP-based FG-ZS-SBIR · Sketchy | Top-1: 10.69 -> 28.68 | Helped |
| Use patch-shuffling loss (ablation) | CLIP-based FG-ZS-SBIR · Sketchy | Top-1: 25.18 -> 28.68; 3.15% | Helped |
| Use relative-distance f-divergence regularization (ablation) | CLIP-based FG-ZS-SBIR · Sketchy | Top-1: 24.93 -> 28.68; 3.75% | Helped |
| Use handcrafted text prompts instead of learned text prompts (ablation) | CLIP-based ZS-SBIR and FG-ZS-SBIR · Sketchy | Top-1: 2.36% | Helped |
| Use CLIP text-encoder embeddings instead of Word2vec embeddings (ablation) | CLIP-based ZS-SBIR and FG-ZS-SBIR · Sketchy | Top-1: 4.57% | Helped |
| Increase visual prompt count from 1 to 3 (ablation) | CLIP-based ZS-SBIR and FG-ZS-SBIR · Sketchy | Top-1: 26.15% -> 28.68% | Helped |
| Increase visual prompt count from 3 to 4 (ablation) | CLIP-based ZS-SBIR and FG-ZS-SBIR · Sketchy | Top-1: 28.68% -> 28.26% | Hurt |
| Use a 2×2 patch grid rather than a 3×3 grid (ablation) | CLIP-based FG-ZS-SBIR · Sketchy | Top-1: 1.2% | Helped |
| Use sketch queries rather than keyword-based CLIP retrieval | ZS-SBIR · Sketchy (extended) | mAP: 0.523 -> 0.723 | Helped |
Closest papers by meaning
- Asymmetric Mutual Alignment for Unsupervised Zero-Shot Sketch-Based Image Retrieval (AAAI 2024 · similarity 0.91)
- You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval (CVPR 2024 · similarity 0.90)
- Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers (CVPR 2024 · similarity 0.90)
- ARNet: Self-Supervised FG-SBIR with Unified Sample Feature Alignment and Multi-Scale Token Recycling (AAAI 2025 · similarity 0.89)
- SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models (CVPR 2025 · similarity 0.89)
- Sketch Down the FLOPs: Towards Efficient Networks for Human Sketch (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.