AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Editing models with task arithmetic

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi

ICLR 2023 · Architectures and efficiency › Pruning and efficient tuning · approach: Parameter-efficient transfer analysis · 31 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Negate the task vector fine-tuned for each target image taskCLIP ViT-L/14 · Eight image-classification tasks (average)target accuracy (%): 64.8 -> 19.0Hurt
Negate the task vector fine-tuned for each target image taskCLIP ViT-L/14 · ImageNet control taskaccuracy (%): 75.5 -> 72.9No clear effect
Use a random vector with the task vector's layer-wise magnitude instead of the task vector (ablation)CLIP image classifier · Image-classification target taskstarget accuracy: 64.8 -> 60.9No clear effect
Use gradient ascent rather than negating the task vectorCLIP ViT-L/14 · ImageNet control taskaccuracy (%): 75.5 -> 16.3Hurt
Negate the toxicity task vectorGPT-2 Large · Toxic generationstoxic generations (%): 4.8 -> 0.8Improved
Negate the toxicity task vectorGPT-2 Large · WikiText-103perplexity: 16.4 -> 16.9No clear effect
Fine-tune on non-toxic examples instead of negating the toxicity task vector (ablation)GPT-2 Large · Toxic generationstoxic generations (%): 4.8 -> 1.8Mixed
Add task vectors for two image-classification tasksCLIP image classifier · Pairs of eight image-classification tasksnormalized accuracy: 98.9%Improved
Add task vectors from external checkpointsFine-tuned T5-base · GLUE averageaccuracy: 78.1 -> 78.6Improved
Use task analogies with auxiliary labels and unlabeled data from both domainsT5-small · Yelp sentiment analysisaccuracy: 88.6 -> 89.9Improved
Use task analogies with auxiliary labels and unlabeled data from both domainsT5-small · Amazon sentiment analysisaccuracy: 87.9 -> 89.0Improved
Combine task vectors from related subpopulations using an analogyCLIP image classifier · Four image subpopulationsaccuracy: +3.4 percentage pointsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy