AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs

Muhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Sivan Doveh, Jakub Micorek, Mateusz Kozinski, Hilde Kuehne, Horst Possegger

ECCV 2024 · Vision-language and multimodal › CLIP and zero-shot · approach: Adapting VLMs for zero-shot classification · 19 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
remove dataset metadata from the meta prompt (ablation)MPVR · EuroSATtop-1 accuracy: 55.6 -> 42.0Hurt
add class names to the stage-1 meta prompt (ablation)MPVR · EuroSATtop-1 accuracy: 55.6 -> 53.5Hurt
use two-stage meta-prompting instead of one-step generation (ablation)MPVR · EuroSATtop-1 accuracy: 51.2 -> 55.6Improved
generate stage-2 category prompts instead of only populating stage-1 templates (ablation)MPVR · EuroSATtop-1 accuracy: 47.2 -> 55.6Improved
stage-1 query templates alone vs CLIP dataset-specific templates (ablation)CLIP ViT-B/32 · EuroSATtop-1 accuracy: 45.8 -> 47.2Improved
ensemble CLIP dataset-specific templates with LLM prompts in embedding space (ablation)MPVR · Mean over 20 datasets (ViT-B/32)top-1 accuracy: 62.9 -> 57.0 (GPT vs GPT+Temp)Hurt
ensemble GPT and Mixtral prompts in embedding space (ablation)MPVR · Mean over 20 datasets (ViT-B/32)top-1 accuracy: 62.9 -> 63.0Improved
ensemble in probability space instead of embedding space (ablation)MPVR (GPT) · 4-dataset mean (ViT-B/32)top-1 accuracy: 61.1 -> 60.0No clear effect
swap in-context dataset to Flowers or CUBS (ablation)MPVR · DTDtop-1 accuracy: variance of +-0.71No clear effect
use MPVR prompts with CLIP instead of LLaVA-1.6 for classificationCLIP ViT-B/32 · 5-dataset meantop-1 accuracy: 30.0 -> 65.5Improved
add MPVR prompts to MetaCLIP ViT-L/14MetaCLIP ViT-L/14 · Mean over 20 datasetstop-1 accuracy: 71.0 -> 74.3Improved
add MPVR prompts to SigLIP and OpenCLIPOpenCLIP, SigLIP · Mean over 20 datasetstop-1 accuracy: 64.4 -> 67.8 (OpenCLIP); 70.8 -> 72.4 (SigLIP)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy