Research map / Label-efficient and robust learning / Recognition and long-tail
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, Yoon Kim
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| transfer a shared prompt learned by multitask prompt decomposition and distillation instead of randomly initialized prompts | prompt tuning · GLUE | average score: 13% | Improved |
| transfer a shared prompt learned by multitask prompt decomposition and distillation instead of randomly initialized prompts | prompt tuning · SuperGLUE | average score: 16% | Improved |
| use a shared transferable prompt with low-rank task-specific updates rather than vanilla prompt tuning | prompt tuning · MRQA | average F1: +2.8% | Improved |
| use a shared transferable prompt with low-rank task-specific updates rather than vanilla prompt tuning | prompt tuning · Others | average accuracy: +13.5% | Improved |
| add prompt decomposition into shared and task-specific components (ablation) | MPT source prompts without distillation · SuperGLUE | average score: 69.5 -> 73.0 | Improved |
| add prompt distillation from separately trained source prompts (ablation) | shared vanilla source prompt · SuperGLUE | average score: 69.5 -> 70.6 | Improved |
| combine prompt decomposition with prompt distillation (ablation) | MPT source prompts · SuperGLUE | average score: 74.1 | Improved |
| add hidden-state matching to logits-based distillation (ablation) | MPT distillation objective · SuperGLUE | average score: 73.7% vs. 74.1% | Improved |
| use logits and hidden-state distillation instead of direct prompt-distance MSE (ablation) | MPT distillation objective · SuperGLUE | average score: 73.6% vs. 74.1% | Improved |
| increase prompt length from 100 to 300 (ablation) | MPT · SuperGLUE | accuracy: 76.8% | Improved |
| increase prompt length from 300 to 400 (ablation) | MPT · SuperGLUE | accuracy: absolute 1.8% drop | Hurt |
| update both the shared component and task-specific vectors during adaptation (ablation) | MPT target adaptation · SuperGLUE | average score: 62.5% and 71.3% | Improved |
Closest papers by meaning
- Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models (ICLR 2024 · similarity 0.91)
- All You Need is One: Capsule Prompt Tuning with a Single Vector (NeurIPS 2025 · similarity 0.91)
- EPT: Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion (AAAI 2025 · similarity 0.89)
- DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning (ICLR 2024 · similarity 0.88)
- Revisiting the Power of Prompt for Visual Tuning (ICML 2024 · similarity 0.88)
- Joint Scheduling of Causal Prompts and Tasks for Multi-Task Learning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
