Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation
Marco Mistretta, Alberto Baldrati, Marco Bertini, Andrew D. Bagdanov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add unsupervised KDPL distillation to CoOp (RN50) | CoOp · Cross-dataset transfer avg | accuracy: 55.28 -> 57.59 | Improved |
| add KDPL to CoOp (RN50) for unseen classes | CoOp · Unseen-class generalization avg | novel accuracy: 60.82 -> 63.80 | Improved |
| add KDPL to CoCoOp (RN50) for domain generalization | CoCoOp · Domain generalization avg | accuracy: 43.38 -> 43.06 | Hurt |
| add KDPL to PromptSRC (ViT-B/32) for domain generalization | PromptSRC · Domain generalization avg | accuracy: 50.67 -> 50.55 | No clear effect |
| add KDPL to VPT (ViT-B/32) for cross-dataset transfer | VPT · Cross-dataset transfer avg | accuracy: 59.79 -> 59.82 | No clear effect |
| add KDPL to MaPLe (ViT-B/32) for unseen classes | MaPLe · Unseen-class generalization avg | novel accuracy: 69.79 -> 71.41 | Improved |
| add KDPL (replacing label supervision) on base classes | CoOp (RN50) · Base-to-novel, base accuracy avg | base accuracy: 77.13 -> 73.66 | Hurt |
| use distillation from teacher logits instead of teacher pseudo-labeling (UPL*) (ablation) | CoOp (RN50) · Cross-dataset avg | accuracy: 56.24 -> 57.59 | Improved |
| select classes by teacher probabilities instead of random sampling (POMP*) (ablation) | CA-KDPL · Cross-dataset avg | accuracy: 57.35 -> 57.69 | Improved |
| use symmetric KL instead of forward-only KL in class agnostic setting (ablation) | CA-KDPL · Cross-dataset avg | accuracy: 56.39 -> 57.69 | Improved |
| use symmetric KL instead of forward KL in label-agnostic setting (ablation) | CoOp + KDPL · Source and domain generalization | accuracy: Source 63.13 -> 62.73; AVG2 57.33 -> 57.59 | Mixed |
| use smaller teacher (B/16 or L/14) instead of ViT-H/14 (ablation) | CoOp + KDPL (RN50) · Unseen-class generalization | novel accuracy: 63.80 -> 67.57 (B/16) | Improved |
Closest papers by meaning
- PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.92)
- Weighted Multi-Prompt Learning with Description-free Large Language Model Distillation (ICLR 2025 · similarity 0.90)
- Learning to Prompt with Text Only Supervision for Vision-Language Models (AAAI 2025 · similarity 0.90)
- Understanding prompt engineering may not require rethinking generalization (ICLR 2024 · similarity 0.90)
- OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP (CVPR 2025 · similarity 0.90)
- A Similarity Paradigm Through Textual Regularization Without Forgetting (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
