Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Multitask Prompted Training Enables Zero-Shot Task Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add multitask prompted training to T5+LM | T5+LM · 11 held-out datasets across four tasks | accuracy | Helped |
| Add multitask prompted training | T5+LM · LAMBADA, standard format | accuracy: 6.2% -> 18.7% | Helped |
| Evaluate with a cloze-style prompted form (ablation) | T0 · LAMBADA | accuracy: 18.7% to 27.8% | Helped |
| Use the GPT-3 dataset variant instead of the official LAMBADA dataset (ablation) | T0 · LAMBADA, cloze-style format | accuracy: 27.8% to 40.5% | Helped |
| Use a wider range of training prompts per dataset (ablation) | T0 · 11 held-out datasets | median accuracy: increases for 8/11 datasets when p increases from 1 to an average of 5.7; increases for 9/11 when including all prompts | Helped |
| Use a wider range of training prompts per dataset (ablation) | T0 · 11 held-out datasets | interquartile range: decreases for 7/11 datasets at p=5.7; decreases for 8/11 datasets with all prompts | Mixed |
| Include prompts that do not correspond to the dataset's original task (ablation) | T0 · 11 held-out datasets | median accuracy: increases for 9/11 datasets | Helped |
| Increase the number of training datasets from 39 to 49 (ablation) | T0 to T0+ · five held-out datasets | median accuracy: increases for all 5 held-out datasets | Helped |
| Increase the number of training datasets from 39 to 49 (ablation) | T0 to T0+ · five held-out datasets | interquartile range: decreases for 1 out of 5 datasets | Mixed |
| Increase the number of training datasets from 49 to 55 (ablation) | T0+ to T0++ · five held-out datasets | median accuracy: increases for all datasets | Helped |
| Increase the number of training datasets from 49 to 55 (ablation) | T0+ to T0++ · five held-out datasets | interquartile range: decreases for 2 out of 5 datasets | Mixed |
| Remove instructions from the prompt (ablation) | T0 · HellaSwag | median accuracy: 33.65% to 57.93% | Helped |
Closest papers by meaning
- Not All Tasks Are Born Equal: Understanding Zero-Shot Generalization (ICLR 2023 · similarity 0.91)
- What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? (ICML 2022 · similarity 0.90)
- Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (NeurIPS 2022 · similarity 0.89)
- Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models (ICLR 2024 · similarity 0.88)
- Task Contamination: Language Models May Not Be Few-Shot Anymore (AAAI 2024 · similarity 0.87)
- Large (Vision) Language Models are Unsupervised In-Context Learners (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.