Research map / Vision-language and multimodal / CLIP and zero-shot
Multimodal Few-Shot Learning with Frozen Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| keep language model frozen instead of finetuning it (ablation) | Frozen · VQAv2 val | accuracy (n=4): 29.2 -> 38.2 | Helped |
| keep language model frozen instead of finetuning it (ablation) | Frozen · OKVQA | accuracy (n=4): 4.6 -> 12.6 | Helped |
| use pretrained LM instead of training whole system from scratch (ablation) | Frozen · VQAv2 val | accuracy (n=0): 0.0 -> 29.5 | Helped |
| provide real image to visual encoder instead of blacked-out image (ablation) | Frozen · VQAv2 val | accuracy (n=4): 33.3 -> 38.2 | Helped |
| add in-context shots (0 to 4) | Frozen · VQAv2 val | accuracy: 29.5 -> 38.2 | Helped |
| scale LM from 400M to 7B (ablation) | Frozen · OKVQA | accuracy (n=4): 6.6 -> 12.6 | Helped |
| add task induction text (ablation) | Frozen · Open-Ended miniImageNet 2-way | accuracy (1 inner shot): 29.0 -> 53.4 | Helped |
| more inner-shots (1 to 5) with different exemplars (ablation) | Frozen · Open-Ended miniImageNet 2-way | accuracy: 53.4 -> 58.9 | Helped |
| repeat a single exemplar (1 to 5 repeats) instead of new exemplars (ablation) | Frozen · Open-Ended miniImageNet 2-way | accuracy: 51.1 -> 58.5 | Helped |
| move from 2-way to 5-way classification | Frozen · Open-Ended miniImageNet | accuracy: 58.9 -> 21.3 | Hurt |
| more inner-shots (0 to 5) | Frozen · Fast-VQA | accuracy: 1.6 -> 7.9 | Helped |
| visual prefix length of 2 tokens versus 1 or 4 (ablation) | Frozen · VQAv2 val | accuracy | Helped |
Closest papers by meaning
- Context-Aware Multimodal Pretraining (CVPR 2025 · similarity 0.90)
- Frozen Transformers in Language Models Are Effective Visual Encoder Layers (ICLR 2024 · similarity 0.89)
- Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning (ICLR 2025 · similarity 0.89)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Emu: Generative Pretraining in Multimodality (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.