Research map / Architectures and efficiency / Pruning and efficient tuning
Editing models with task arithmetic
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Negate the task vector fine-tuned for each target image task | CLIP ViT-L/14 · Eight image-classification tasks (average) | target accuracy (%): 64.8 -> 19.0 | Hurt |
| Negate the task vector fine-tuned for each target image task | CLIP ViT-L/14 · ImageNet control task | accuracy (%): 75.5 -> 72.9 | No clear effect |
| Use a random vector with the task vector's layer-wise magnitude instead of the task vector (ablation) | CLIP image classifier · Image-classification target tasks | target accuracy: 64.8 -> 60.9 | No clear effect |
| Use gradient ascent rather than negating the task vector | CLIP ViT-L/14 · ImageNet control task | accuracy (%): 75.5 -> 16.3 | Hurt |
| Negate the toxicity task vector | GPT-2 Large · Toxic generations | toxic generations (%): 4.8 -> 0.8 | Improved |
| Negate the toxicity task vector | GPT-2 Large · WikiText-103 | perplexity: 16.4 -> 16.9 | No clear effect |
| Fine-tune on non-toxic examples instead of negating the toxicity task vector (ablation) | GPT-2 Large · Toxic generations | toxic generations (%): 4.8 -> 1.8 | Mixed |
| Add task vectors for two image-classification tasks | CLIP image classifier · Pairs of eight image-classification tasks | normalized accuracy: 98.9% | Improved |
| Add task vectors from external checkpoints | Fine-tuned T5-base · GLUE average | accuracy: 78.1 -> 78.6 | Improved |
| Use task analogies with auxiliary labels and unlabeled data from both domains | T5-small · Yelp sentiment analysis | accuracy: 88.6 -> 89.9 | Improved |
| Use task analogies with auxiliary labels and unlabeled data from both domains | T5-small · Amazon sentiment analysis | accuracy: 87.9 -> 89.0 | Improved |
| Combine task vectors from related subpopulations using an analogy | CLIP image classifier · Four image subpopulations | accuracy: +3.4 percentage points | Improved |
Closest papers by meaning
- When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers (ICLR 2025 · similarity 0.90)
- Training-Free Task Vectors for LLM Behavioral Control (NeurIPS 2026 · similarity 0.88)
- Knowledge Composition using Task Vectors with Learned Anisotropic Scaling (NeurIPS 2024 · similarity 0.86)
- AdaMerging: Adaptive Model Merging for Multi-Task Learning (ICLR 2024 · similarity 0.86)
- Efficient Model Editing with Task-Localized Sparse Fine-tuning (ICLR 2025 · similarity 0.86)
- Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
