Research map / Vision-language and multimodal / CLIP and zero-shot
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a task-specific prompt | GPT-2 fixed features f(s) · SST | accuracy (%): 87.5 -> 89.4 | Helped |
| Add a task-specific prompt | Conditional mean features · SST | accuracy (%): 83.3 -> 87.3 | Helped |
| Add a task-specific prompt | GPT-2 fixed features f(s) · SST fine | accuracy (%): 49.2 -> 49.4 | No clear effect |
| Add a task-specific prompt | Conditional mean features · SST fine | accuracy (%): 43.5 -> 48.6 | Helped |
| Add a task-specific prompt | GPT-2 fixed features f(s) · AG News | accuracy (%): 90.7 -> 91.1 | Helped |
| Add a task-specific prompt | Conditional mean features · AG News | accuracy (%): 84.6 -> 88.2 | Helped |
| Use conditional mean features instead of f(s) | GPT-2 · SST | accuracy (%): 87.5 -> 83.3 | Hurt |
| Use conditional mean features instead of f(s) | GPT-2 · AG News | accuracy (%): 90.7 -> 84.6 | Hurt |
| Use a random projection of LM probabilities instead of conditional mean features | GPT-2 features · SST | accuracy (%): 83.3 -> 67.5 | Hurt |
| Use a random projection of LM probabilities instead of conditional mean features | GPT-2 features · AG News | accuracy (%): 84.6 -> 58.5 | Hurt |
| Use ≤30 task-relevant token probabilities rather than full GPT-2 features | GPT-2 · SST | accuracy (%): 87.5 -> 82.6 | Hurt |
| Use ≤30 task-relevant token probabilities rather than full GPT-2 features | GPT-2 · AG News | accuracy (%): 90.7 -> 83.8 | Hurt |
Closest papers by meaning
- Language Model Cascades: Token-Level Uncertainty And Beyond (ICLR 2024 · similarity 0.87)
- Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima (NeurIPS 2026 · similarity 0.87)
- Idiosyncrasies in Large Language Models (ICML 2025 · similarity 0.87)
- Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training (NeurIPS 2025 · similarity 0.87)
- Language models scale reliably with over-training and on downstream tasks (ICLR 2025 · similarity 0.86)
- Enhancing Elusive Clues in Knowledge Learning by Contrasting Attention of Language Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.