Research map / Architectures and efficiency / Pruning and efficient tuning
TIES-Merging: Resolving Interference When Merging Models
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TIES-Merging instead of Task Arithmetic, with validation | Merged ViT-B/32 · 8 vision tasks | average accuracy: 70.1 -> 73.6 | Improved |
| Use TIES-Merging instead of Task Arithmetic, no validation | Merged ViT-B/32 · 8 vision tasks | average accuracy: 60.4 -> 72.4 | Improved |
| Use TIES-Merging instead of Task Arithmetic, with validation | Merged ViT-L/14 · 8 vision tasks | average accuracy: 84.5 -> 86.0 | Improved |
| Use TIES-Merging instead of Task Arithmetic, with validation | Merged T5-Large · 7 NLP tasks | average accuracy: 73.3 -> 76.9 | Improved |
| Use TIES-Merging instead of Task Arithmetic, no validation | Merged T5-Base · 7 NLP tasks | average accuracy: 73.2 -> 69.7 | Hurt |
| Use TIES-Merging instead of Task Arithmetic, with validation | Merged (IA)3 models · 11 tasks | average accuracy: 63.9 -> 66.4 | Improved |
| Use TIES-Merging instead of Task Arithmetic | Merged T5-Large · OOD six held-out tasks | average accuracy: 33.3 -> 40.4 | Improved |
| Use TIES-Merging instead of RegMean | Merged T5-Base · OOD six held-out tasks | average accuracy: 34.3 -> 35.3 | Improved |
| Oracle multitask sign vector instead of elected sign | TIES (IA)3 · 11 tasks | average accuracy: 66.4 -> 72.0 | Improved |
| Sign vector from 32-sample multitask init from mean | TIES (IA)3 · 11 tasks | average accuracy: 66.4 -> 67.7 | Improved |
| Sign vector from 32-sample multitask trained from scratch | TIES (IA)3 · 11 tasks | average accuracy: 66.4 -> 66.5 | No clear effect |
| Add scaling lambda (vs lambda=1) to TIES (ablation) | TIES T5-Base · validation set | accuracy: 2.5% drop without it | Improved |
Later papers that compared with this method
16 later accepted papers report TIES-Merging as a baseline in their results tables; 16 of them report a better number on most of the results they share with it.
- Saliency-Aware Model Merging (ICML 2026 · better on 4 of 4 shared results)
- MergOPT: A Merge-Aware Optimizer for Robust Model Merging (ICLR 2026 · better on 4 of 4 shared results)
- Model Merging in the Essential Subspace (CVPR 2026 · better on 2 of 2 shared results)
- When Model Merging Breaks Routing: Training-Free Calibration for MoE (ICML 2026 · better on 1 of 1 shared result)
- DuetMerging: Synergizing Dynamic and Static Strategies for Mitigating Task Interference in Model Merging (CVPR 2026 · better on 1 of 1 shared result)
- Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging (ECCV 2026 · better on 1 of 1 shared result)
- FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization (ICCV 2025 · better on 5 of 5 shared results)
- Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models (TPAMI 2025 · better on 4 of 4 shared results)
8 more not listed.
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Parameter Competition Balancing for Model Merging (NeurIPS 2024 · similarity 0.91)
- Whoever Started the interference Should End It: Guiding Data-Free Model Merging via Task Vectors (ICML 2025 · similarity 0.90)
- Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging (NeurIPS 2024 · similarity 0.90)
- MergeBench: A Benchmark for Merging Domain-Specialized LLMs (NeurIPS 2025 · similarity 0.90)
- Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent (ICML 2025 · similarity 0.89)
- AdaMerging: Adaptive Model Merging for Multi-Task Learning (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
