Research map / Learning theory and algorithms / Combinatorial and search optimization
Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Pareto Merging, which formulates model merging as multi-objective optimization and learns, in one process, a Pareto set of merged ViT models via a shared merged base plus a low-rank tensor conditioned on user preference vectors.
Problem
Existing model merging methods output a single merged model, which cannot satisfy users with different preferences over the tasks of the source models due to inter-task conflicts.
Approach
Treat each source task's performance (or unlabeled-data entropy) as an objective, and parameterize the merged model as a preference-independent base plus a preference-dependent low-rank tensor term (G x A x B x gamma), trained with smooth Tchebycheff scalarization over Dirichlet-sampled preferences, so that a single merging process yields a continuous Pareto set with minimal extra parameters.
Key results
- Merging eight ViT-B/32 models, AdaMerging+TIES+PM (priority) reaches 85.9% average accuracy versus 81.1% for AdaMerging+TIES.
- Task Arithmetic+PM (priority) improves average accuracy from 69.1% to 71.0% when merging eight ViT-B/32 models; TIES-Merging+PM (priority) improves from 72.4% to 74.1%.
- On unseen MNIST/EuroSAT, AdaMerging+PM raises average accuracy from 70.0% to 74.7% over AdaMerging.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add priority-preference Pareto Merging to Task Arithmetic | Task Arithmetic · 8-task ViT-B/32 | average accuracy: 69.1 -> 71.0 | Helped |
| Add priority-preference Pareto Merging to TIES-Merging | TIES-Merging · 8-task ViT-B/32 | average accuracy: 72.4 -> 74.1 | Helped |
| Add equal-preference Pareto Merging to AdaMerging | AdaMerging · 8-task ViT-B/32 | average accuracy: 80.1 -> 84.9 | Helped |
| Add priority-preference Pareto Merging to AdaMerging+TIES | AdaMerging+TIES · 8-task ViT-B/32 | average accuracy: 81.1 -> 85.9 | Helped |
| Add Pareto Merging and select a model from its learned set | AdaMerging · Unseen MNIST and EuroSAT, six merged models | average accuracy: 70.0 -> 74.7 | Helped |
| Add Pareto Merging and select a model from its learned set | AdaMerging+TIES · Unseen MNIST and EuroSAT, six merged models | average accuracy: 68.7 -> 75.7 | Helped |
| Add Pareto Merging and select a model from its learned set | TIES Merging · Unseen RESISC45 and SVHN, six merged models | average accuracy: 53.9 -> 55.5 | Helped |
| Add Pareto Merging and select a model from its learned set | AdaMerging+TIES · Unseen RESISC45 and SVHN, six merged models | average accuracy: 58.5 -> 58.8 | Helped |
| Vary the tensor rank r (ablation) | AdaMerging+PM · RESISC45 and GTSRB, 2-model merge | test accuracy | Mixed |
| Replace the tensor-based preference module with either outer-product variant (ablation) | Pareto Merging · RESISC45 and GTSRB, 2-model merge | test accuracy | Hurt |
Surprising findings
- Rewarded Soups' simple weighted averaging performs much worse than other merging methods in the middle of the trade-off region.
Limitations
- Experiments are limited to image classification with CLIP ViT backbones; the authors note extension to large language models is future work.
- PM requires slightly more computation time than the baselines it augments.
- Only two representative preference types (equal and priority) are reported for eight-model merging, since showing all preferences is infeasible.
Experimental setup
Datasets: SUN397 Cars RESISC45 EuroSAT SVHN GTSRB MNIST DTD
Benchmarks: SUN397 Cars RESISC45 EuroSAT SVHN GTSRB MNIST DTD
Metrics: test accuracy average accuracy GPU hours parameter count
Baselines: Weight Averaging Fisher Merging RegMean DARE MAP Rewarded Soups Task Arithmetic TIES-Merging AdaMerging AdaMerging+TIES Individual Traditional MTL
Builds on: Task Arithmetic AdaMerging TIES-Merging Rewarded Soups MAP
Closest papers by meaning
- HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models (NeurIPS 2025 · similarity 0.87)
- User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimization (AAAI 2025 · similarity 0.87)
- FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning (NeurIPS 2024 · similarity 0.87)
- Preference-Guided Diffusion for Multi-Objective Offline Optimization (NeurIPS 2025 · similarity 0.86)
- Mixture-Trained Merging for Unified Multi-Objective Models (NeurIPS 2026 · similarity 0.85)
- Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.