Research map / Vision-language and multimodal / CLIP and zero-shot
$\pi$-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation
Chengyue Wu, Teng Wang, Yixiao Ge, Zeyu Lu, Ruisong Zhou, Ying Shan, Ping Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Interpolate target adapter with experts retrieved from similar tasks | Adapter · Vision tasks (8 datasets), 16-shot | AVG accuracy: +4.55% | Improved |
| Interpolate target adapter with experts retrieved from similar tasks | Adapter · GLUE, zero-shot | AVG: +6.05 | Improved |
| Interpolate similar-task adapters | Adapter · GLUE, full data on T5-Base | AVG: +1.75 | Improved |
| Use π-tuning with prompt experts | Prompt Tuning · RefCOCO, RefCOCO+, and RefCOCOg | accuracy: +1.94% average accuracy | Improved |
| Use π-tuning with LoRA experts | LoRA · RefCOCO, RefCOCO+, and RefCOCOg | accuracy: +4.20% average accuracy | Improved |
| Randomly initialize auxiliary experts instead of initializing from trained similar-task experts (ablation) | π-Adapter · RefCOCO val | ACC@0.5: 89.95 vs 90.49 | Hurt |
| Tune only interpolation weights instead of tuning both expert parameters and weights (ablation) | π-Adapter · RefCOCO val | ACC@0.5: 90.67 vs 90.49 | Hurt |
| Increase the number of auxiliary experts beyond the best-performing count (ablation) | π-Tuning · RefCOCO, RefCOCO+, and RefCOCOg | accuracy | Mixed |
Closest papers by meaning
- Re-Imagining Multimodal Instruction Tuning: A Representation View (ICLR 2025 · similarity 0.89)
- Understanding the Transferability of Representations via Task-Relatedness (NeurIPS 2024 · similarity 0.89)
- Understanding Task Transfer in Vision-Language Models (CVPR 2026 · similarity 0.89)
- A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter (AAAI 2025 · similarity 0.89)
- Exploring the Transferability of Visual Prompting for Multimodal Large Language Models (CVPR 2024 · similarity 0.88)
- TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
