Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Fine-Tune Once, Reuse Across Models: Bayesian Task-Update Factors and Approximations
Siyang Guo, Junbo Wang, Zibin Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| transfer Bayesian task-update factor (Fisher/Laplace) instead of adding parameter difference | RoBERTa new bases (codebert/4m) · 7 text tasks (average) | accuracy (%): 59.93 -> 80.24 | Improved |
| add parameter-space task vector transfer to new base | codebert on MNLI and RTE · MNLI and RTE | accuracy (%): 35.91 -> 35.44 (MNLI) | Hurt |
| add permutation re-basin step (TransFusion) to parameter delta transfer | Param-delta · roberta-base to legal/mental | accuracy (%): 82.57 -> 82.57 (SST-2 legal) | No clear effect |
| transfer multiple task updates jointly (sum increments) | BTransfer · 4-task joint transfer | normalized gain: 0.473 -> 0.761 vs Param-delta | Improved |
| increase Fisher sample size from 50 to about 500 (ablation) | BTransfer · 7 text tasks | accuracy (%) | Improved |
| increase Fisher sample size from 1,000 to 10,000 (ablation) | BTransfer · 7 text tasks | accuracy (%) | No clear effect |
| BTransfer to LLM new bases instead of parameter delta | gemma-2-2b-it variants · 5 capability categories | accuracy (%): +6.63 (BG-2B) vs -7.08 for Param-delta over new base | Improved |
| BTransfer on math vs Param-delta on Atlas-2B | BTransfer · Math, Atlas-2B | accuracy (%): 52.62 (Param-delta) vs 47.08 | Hurt |
| BTransfer on CLIP vision models | CLIP ViT-B/16 · SVHN, GTSRB, EuroSAT, DTD | accuracy (%): 43.33 -> 53.48 (GTSRB datacomp) | Improved |
Closest papers by meaning
- Update Your Transformer to the Latest Release: Re-Basin of Task Vectors (ICML 2025 · similarity 0.89)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.87)
- Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks (ICLR 2024 · similarity 0.87)
- Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning (ICLR 2025 · similarity 0.87)
- LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views (ICML 2024 · similarity 0.87)
- Overcoming Generic Knowledge Loss with Selective Parameter Update (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
