AI Paper Atlas

Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

Marco Mistretta, Alberto Baldrati, Marco Bertini, Andrew D. Bagdanov

ECCV 2024 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Robust fine-tuning of zero-shot models · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add unsupervised KDPL distillation to CoOp (RN50)CoOp · Cross-dataset transfer avgaccuracy: 55.28 -> 57.59Improved
add KDPL to CoOp (RN50) for unseen classesCoOp · Unseen-class generalization avgnovel accuracy: 60.82 -> 63.80Improved
add KDPL to CoCoOp (RN50) for domain generalizationCoCoOp · Domain generalization avgaccuracy: 43.38 -> 43.06Hurt
add KDPL to PromptSRC (ViT-B/32) for domain generalizationPromptSRC · Domain generalization avgaccuracy: 50.67 -> 50.55No clear effect
add KDPL to VPT (ViT-B/32) for cross-dataset transferVPT · Cross-dataset transfer avgaccuracy: 59.79 -> 59.82No clear effect
add KDPL to MaPLe (ViT-B/32) for unseen classesMaPLe · Unseen-class generalization avgnovel accuracy: 69.79 -> 71.41Improved
add KDPL (replacing label supervision) on base classesCoOp (RN50) · Base-to-novel, base accuracy avgbase accuracy: 77.13 -> 73.66Hurt
use distillation from teacher logits instead of teacher pseudo-labeling (UPL*) (ablation)CoOp (RN50) · Cross-dataset avgaccuracy: 56.24 -> 57.59Improved
select classes by teacher probabilities instead of random sampling (POMP*) (ablation)CA-KDPL · Cross-dataset avgaccuracy: 57.35 -> 57.69Improved
use symmetric KL instead of forward-only KL in class agnostic setting (ablation)CA-KDPL · Cross-dataset avgaccuracy: 56.39 -> 57.69Improved
use symmetric KL instead of forward KL in label-agnostic setting (ablation)CoOp + KDPL · Source and domain generalizationaccuracy: Source 63.13 -> 62.73; AVG2 57.33 -> 57.59Mixed
use smaller teacher (B/16 or L/14) instead of ViT-H/14 (ablation)CoOp + KDPL (RN50) · Unseen-class generalizationnovel accuracy: 63.80 -> 67.57 (B/16)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy