Research map / Vision-language and multimodal / CLIP and zero-shot
Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs
Muhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Sivan Doveh, Jakub Micorek, Mateusz Kozinski, Hilde Kuehne, Horst Possegger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove dataset metadata from the meta prompt (ablation) | MPVR · EuroSAT | top-1 accuracy: 55.6 -> 42.0 | Hurt |
| add class names to the stage-1 meta prompt (ablation) | MPVR · EuroSAT | top-1 accuracy: 55.6 -> 53.5 | Hurt |
| use two-stage meta-prompting instead of one-step generation (ablation) | MPVR · EuroSAT | top-1 accuracy: 51.2 -> 55.6 | Improved |
| generate stage-2 category prompts instead of only populating stage-1 templates (ablation) | MPVR · EuroSAT | top-1 accuracy: 47.2 -> 55.6 | Improved |
| stage-1 query templates alone vs CLIP dataset-specific templates (ablation) | CLIP ViT-B/32 · EuroSAT | top-1 accuracy: 45.8 -> 47.2 | Improved |
| ensemble CLIP dataset-specific templates with LLM prompts in embedding space (ablation) | MPVR · Mean over 20 datasets (ViT-B/32) | top-1 accuracy: 62.9 -> 57.0 (GPT vs GPT+Temp) | Hurt |
| ensemble GPT and Mixtral prompts in embedding space (ablation) | MPVR · Mean over 20 datasets (ViT-B/32) | top-1 accuracy: 62.9 -> 63.0 | Improved |
| ensemble in probability space instead of embedding space (ablation) | MPVR (GPT) · 4-dataset mean (ViT-B/32) | top-1 accuracy: 61.1 -> 60.0 | No clear effect |
| swap in-context dataset to Flowers or CUBS (ablation) | MPVR · DTD | top-1 accuracy: variance of +-0.71 | No clear effect |
| use MPVR prompts with CLIP instead of LLaVA-1.6 for classification | CLIP ViT-B/32 · 5-dataset mean | top-1 accuracy: 30.0 -> 65.5 | Improved |
| add MPVR prompts to MetaCLIP ViT-L/14 | MetaCLIP ViT-L/14 · Mean over 20 datasets | top-1 accuracy: 71.0 -> 74.3 | Improved |
| add MPVR prompts to SigLIP and OpenCLIP | OpenCLIP, SigLIP · Mean over 20 datasets | top-1 accuracy: 64.4 -> 67.8 (OpenCLIP); 70.8 -> 72.4 (SigLIP) | Improved |
Closest papers by meaning
- SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models (CVPR 2025 · similarity 0.90)
- Learning to Prompt with Text Only Supervision for Vision-Language Models (AAAI 2025 · similarity 0.90)
- Large Language Models are Good Prompt Learners for Low-Shot Image Classification (CVPR 2024 · similarity 0.90)
- Weighted Multi-Prompt Learning with Description-free Large Language Model Distillation (ICLR 2025 · similarity 0.90)
- Learning to Learn Better Visual Prompts (AAAI 2024 · similarity 0.89)
- PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
