Research map / Architectures and efficiency / Pruning and efficient tuning

Platonic Task Arithmetic

NeurIPS 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Parameter-efficient transfer analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Universal Task Descriptors (image-prompt affinity-shift matrices) to transfer and compose fine-tuning-induced task changes across heterogeneous vision-language models, via a closed-form ridge-solved embedding operator folded into the last layer or a per-edit low-rank adapter.

Problem

Weight-space task arithmetic only works within one model because parameter-space task vectors do not share a coordinate system, so task knowledge cannot be transferred or composed across architectures.

Approach

Measure each fine-tuning's effect as a model-independent N x M matrix of changes in image-prompt cosine affinities (UTD), then transfer it to a target by solving a ridge least-squares problem for a linear embedding operator folded into the target's last layer; operators from different sources and tasks compose by addition.

Key results

• Closed-form composition of two tasks from two other models lifts Task A and Task B by 18.11 and 11.63 points over zero-shot, retaining 80% and 74% of the Oracle gain.
• Negation at lambda=1 over 240 cross-model cells removes 35.2 points of the targeted task for 4.5 points of CIFAR-100 cost.
• Adapter route on 168 cells reaches 79.50 and 76.70 on the two composed tasks against 79.05 and 72.65 for the closed form.

Surprising findings

• Descriptors of the same task from different models agree at cosine 0.47, while different tasks from one model agree at 0.23, so descriptors encode which task was learned.
• Transfer retains most of the gain even though a source's operator agrees with the target's own-descriptor operator only at cosine 0.39.

Limitations

• Hypothesis of cross-model consistency is supported rather than derived, with residual comparable in norm to the shared component.
• Both realizations need semantic prompts and unlabeled probe images per task, so they are label-free but not data-free and exclude single-modality models.
• Interference between operators from different tasks has no smallness guarantee; transfer gains at most about two points on Cars and SUN397.

Experimental setup

Datasets: EuroSAT DTD Cars GTSRB MNIST RESISC45 SVHN SUN397 AID UCMerced ESC-50 GTZAN

Benchmarks: EuroSAT DTD Cars GTSRB MNIST RESISC45 SVHN SUN397 CIFAR-100 ImageNet AID UCMerced

Metrics: accuracy retained gain percent

Baselines: Weight-space task arithmetic Weight-space transport GradFix Theseus Gradient ascent Distillation Supervised tuning Random-UTD

Builds on: task arithmetic LoRA Platonic Representation Hypothesis relational distillation

Closest papers by meaning

• Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (NeurIPS 2026 · similarity 0.88)
• Position: The Platonic Representation Hypothesis (ICML 2024 · similarity 0.87)
• Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities (NeurIPS 2026 · similarity 0.87)
• Disentangling Representations through Multi-task Learning (ICLR 2025 · similarity 0.86)
• PLATO: Pointer Learner for Agent and Task Openness (NeurIPS 2026 · similarity 0.86)
• Attention as a Hypernetwork (ICLR 2025 · similarity 0.85)