AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Distilling Linearized Behavior into Non-linear Fine-Tuning for Effective Task Arithmetic

Thomas Sommariva, Francesca Morandi, Simone Calderara, Angelo Porrello

ICML 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Parameter-efficient transfer analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
distill linearized teacher activations into non-linear student with curvature regularization (DELTA) vs plain non-linear FTCLIP ViT-B/32 non-linear fine-tuning · 8-Vision task addition (alpha=1.0)absolute accuracy: 32.0 -> 88.3Improved
add linearized teacher distillation on top of curvature regularization (ablation)DELTA · 8-Vision task addition, ViT-B/32mean absolute accuracy: 81.1 -> 88.3Improved
add EK-FAC curvature regularization on top of distillation (ablation)DELTA · 8-Vision task addition, ViT-B/32mean absolute accuracy: 84.2 -> 88.3Improved
add distillation and curvature regularization on T5-Base (ablation)DELTA · 6-NLI task addition, T5-Basemean absolute accuracy: 78.9 -> 82.4 (w/o teacher vs full); 78.5 -> 82.4 (w/o EK-FAC vs full)Improved
add linearized teacher distillation for task negation (ablation)DELTA · 8-Vision task negation, ViT-B/32target accuracy: 14.2 -> 9.6Improved
add curvature regularization for task negation (ablation)DELTA · 8-Vision task negation, ViT-B/32target accuracy: 16.8 -> 9.6Improved
sample teacher along path (alpha sampling) instead of fixing alpha=1 (ablation)DELTA distillation · 8-Vision, linearization error and alpha sweeplinearization error and robustness to alphaImproved
use ImageNet-1k instead of ImageNet-21k as reference dataset for curvature (ablation)DELTA · 8-Vision task addition, ViT-B/32 (alpha=1.0)absolute accuracy: 88.3 -> 87.5Hurt
use PAWS instead of C4 as reference dataset (ablation)DELTA · 6-NLI task addition, T5-Base (alpha=1.0)absolute accuracy: 82.3 -> 80.3Hurt
use QQP instead of C4 as reference dataset (ablation)DELTA · 6-NLI task addition, T5-Base (alpha=1.0)absolute accuracy: 82.3 -> 81.6Hurt
distill linearized teacher into a LoRA student instead of post-hoc merging (Iso-C, TSV-M, Core)LoRA task addition · 8-Vision, ViT-B/32absolute accuracy: 77.9 -> 87.5Improved
DELTA versus linear-regime methods (tau-Jp, TAK) for task negationtask negation · 8-Vision task negation, ViT-B/32target accuracy: 2.8 -> 9.6 (TAK vs DELTA)Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy