AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models

Guillermo Ortiz-Jimenez, Alessandro Favero, Pascal Frossard

NeurIPS 2023 · Architectures and efficiency › Pruning and efficient tuning · approach: Low-rank compression and merging · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune in the tangent space (linearized FT) instead of standard non-linear fine-tuningCLIP ViT-B/32 · task addition, 8 tasksabsolute accuracy: 71.4 -> 76.5Improved
fine-tune in the tangent space instead of non-linear fine-tuningCLIP ViT-L/14 · task addition, 8 tasksabsolute accuracy: 85.1 -> 88.5Improved
fine-tune in the tangent space instead of non-linear fine-tuningCLIP ViT-B/32 · task negation, target accuracytarget accuracy (lower better): 24.0 -> 10.9Improved
fine-tune in tangent space instead of non-linear on ViT-B/16 negationCLIP ViT-B/16 · task negation, target accuracytarget accuracy (lower better): 19.2 -> 11.3Improved
post-hoc linearization of non-linearly fine-tuned task vectors instead of non-linear model (ablation)CLIP ViT-B/32 · task addition, 8 tasksabsolute accuracy: 71.4 -> 57.1Hurt
post-hoc linearization of non-linearly fine-tuned task vectors (ablation)CLIP ViT-B/32 · task addition, 8 tasksnormalized accuracy: 76.5 -> 81.9Improved
post-hoc linearization instead of non-linear model (ablation)CLIP ViT-B/32 · task negation, target accuracytarget accuracy (lower better): 24.0 -> 14.8Improved
use pre-trained instead of random initialization for task vectors (ablation)ViT-B/32 · task addition from initmulti-task accuracy: 5.5 -> 71.4Improved
linearized fine-tuning from random initialization (ablation)ViT-B/32 random init · task additionmulti-task accuracy: 5.5 -> 3.8No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy