Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Variational Task Vector Composition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes variational task vector composition with sample-specific latent composition coefficients, a Spike-and-Slab prior, and a deterministic gated sampling mechanism, evaluated on eight CLIP ViT image classification datasets.
Problem
Task vector composition (task arithmetic) is deterministic and task-level, lacking uncertainty quantification and sample-level adaptivity, and task vector spaces show structural redundancy.
Approach
Treats composition coefficients as latent variables inferred per sample via amortized variational inference, uses a Spike-and-Slab prior for sparsity, and replaces stochastic Bernoulli sampling with a deterministic uncertainty- and importance-based soft gate.
Key results
- Sample-specific probabilistic composition reaches 84.32% average accuracy on ViT-B/32 vs 77.60% for task-level deterministic task arithmetic.
- Adding gated sampling raises average accuracy from 84.69% to 86.04% on ViT-B/32 (Spike-and-Slab with gating), and to 91.54% on ViT-L/14 in the appendix table.
- Applying the gating filter to aTLAS improves its average accuracy from 77.60% to 81.16% on ViT-B/32.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sample-specific rather than task-level probabilistic composition | VTVC · Cars | accuracy: 73.06% -> 75.33% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · DTD | accuracy: 80.64% -> 85.71% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · EuroSAT | accuracy: 96.19% -> 96.33% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · GTSRB | accuracy: 85.66% -> 86.76% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · MNIST | accuracy: 98.65% -> 98.72% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · RESISC45 | accuracy: 90.00% -> 90.98% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · SUN397 | accuracy: 67.95% -> 72.97% | Helped |
| use sample-specific rather than task-level probabilistic composition | VTVC · SVHN | accuracy: 64.23% -> 67.78% | Helped |
| replace Gaussian prior with Spike-and-Slab prior (ablation) | VTVC · Average (eight datasets), ViT-B/32 | accuracy: 84.32% -> 84.69% | Helped |
| add gated sampling (ablation) | VTVC with Spike-and-Slab prior · Average (eight datasets), ViT-B/32 | accuracy: 84.69% -> 86.04% | Helped |
| add gated sampling (ablation) | VTVC with Spike-and-Slab prior · GTSRB, ViT-B/32 | accuracy: 87.17% -> 87.16% | No clear effect |
| apply gated sampling to aTLAS | aTLAS · Average (eight datasets), ViT-B/32 | accuracy: 77.60% -> 81.16% | Helped |
Surprising findings
- Gaussian priors perform slightly better on EuroSAT and RESISC45, and deterministic methods are slightly better on DTD in some comparisons.
Limitations
- Only task addition is studied; other task arithmetic operations are not explored.
- High GPU memory needs limit experiments to ViT-scale models; no language or convolutional architectures tested.
- Method relies on several tuned hyperparameters and regularizers whose sensitivity is only partly analyzed.
Experimental setup
Datasets: Cars DTD EuroSAT GTSRB MNIST RESISC45 SUN397 SVHN
Benchmarks: Cars DTD EuroSAT GTSRB MNIST RESISC45 SUN397 SVHN
Metrics: absolute accuracy gated ratio
Baselines: Task-level task arithmetic aTLAS Gaussian prior variational composition zero-shot CLIP
Builds on: task arithmetic aTLAS CLIP
Closest papers by meaning
- Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning (NeurIPS 2026 · similarity 0.89)
- Knowledge Composition using Task Vectors with Learned Anisotropic Scaling (NeurIPS 2024 · similarity 0.88)
- Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (NeurIPS 2026 · similarity 0.88)
- Scaling can lead to compositional generalization (NeurIPS 2025 · similarity 0.87)
- TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types (ICLR 2025 · similarity 0.86)
- In-Context Compositional Learning vis Sparse Coding Transformer (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.