Research map / Architectures and efficiency / Pruning and efficient tuning
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models
Guillermo Ortiz-Jimenez, Alessandro Favero, Pascal Frossard
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune in the tangent space (linearized FT) instead of standard non-linear fine-tuning | CLIP ViT-B/32 · task addition, 8 tasks | absolute accuracy: 71.4 -> 76.5 | Improved |
| fine-tune in the tangent space instead of non-linear fine-tuning | CLIP ViT-L/14 · task addition, 8 tasks | absolute accuracy: 85.1 -> 88.5 | Improved |
| fine-tune in the tangent space instead of non-linear fine-tuning | CLIP ViT-B/32 · task negation, target accuracy | target accuracy (lower better): 24.0 -> 10.9 | Improved |
| fine-tune in tangent space instead of non-linear on ViT-B/16 negation | CLIP ViT-B/16 · task negation, target accuracy | target accuracy (lower better): 19.2 -> 11.3 | Improved |
| post-hoc linearization of non-linearly fine-tuned task vectors instead of non-linear model (ablation) | CLIP ViT-B/32 · task addition, 8 tasks | absolute accuracy: 71.4 -> 57.1 | Hurt |
| post-hoc linearization of non-linearly fine-tuned task vectors (ablation) | CLIP ViT-B/32 · task addition, 8 tasks | normalized accuracy: 76.5 -> 81.9 | Improved |
| post-hoc linearization instead of non-linear model (ablation) | CLIP ViT-B/32 · task negation, target accuracy | target accuracy (lower better): 24.0 -> 14.8 | Improved |
| use pre-trained instead of random initialization for task vectors (ablation) | ViT-B/32 · task addition from init | multi-task accuracy: 5.5 -> 71.4 | Improved |
| linearized fine-tuning from random initialization (ablation) | ViT-B/32 random init · task addition | multi-task accuracy: 5.5 -> 3.8 | No clear effect |
Closest papers by meaning
- Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic (ICLR 2025 · similarity 0.92)
- Efficient Model Editing with Task-Localized Sparse Fine-tuning (ICLR 2025 · similarity 0.87)
- DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformations (CVPR 2025 · similarity 0.87)
- Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs (ICLR 2025 · similarity 0.87)
- Platonic Task Arithmetic (NeurIPS 2026 · similarity 0.86)
- No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
