Research map / Learning theory and algorithms / Combinatorial and search optimization

Mixture-Trained Merging for Unified Multi-Objective Models

NeurIPS 2026 · Learning theory and algorithms › Combinatorial and search optimization · approach: Multi-objective optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mixture-Trained Merging (MTM), which trains each branch on an objective-biased data mixture, uses qNEHVI Bayesian optimization over merge coefficients to select branch mixtures, and iteratively promotes merged models so that the branches remain mergeable.

Problem

Merging independently trained single-objective experts causes sharp cross-domain degradation and collapse of think/non-think modes, because the experts drift into incompatible weight-space regions; sequential single-trajectory multi-objective post-training is also highly sensitive to data mixtures a

Approach

Train each merge branch on an objective-biased data mixture instead of a pure single-objective dataset so branches stay mergeable, use merged-model evaluations (via the MergeMix linear-merge approximation) as a cheap signal for choosing branch mixtures, and iterate rounds of branch training, MOBO-based merge search, base promotion, and mixture update.

Key results

• On Qwen3-4B-Base, MTM-Multi reaches the highest average (69.74) across GSM8K, MATH, MBPP+, LCB, and IFEval, versus 68.68 for Linear Merge and 67.78 for MergeMix.
• On OLMo-7B, MTM-Multi reaches the highest average (64.10 ± 0.97), versus 60.56 for TIES-Merge and 63.21 for MergeMix.
• In the hybrid OLMo-7B think/no-think setting, Linear Merge produces a near-degenerate length gap between modes, while MTM keeps the modes separated and uses fewer tokens in no-think mode.

Surprising findings

• Linear merging of hard single-objective experts collapses think and no-think modes into one behavior, even though each expert is individually strong.
• Mixture-trained branches merged with MTM transfer their recipe to Qwen2.5-0.5B without re-search and beat size-matched baselines.

Limitations

• Study limited to SFT-stage merging; post-RL training not explored.
• Evaluated only on a small set of objectives (math, code, IF, think/non-think) and two base models.
• Multi-round training with BO search adds substantial compute compared with simple merging, although the paper argues wall-clock cost is comparable to single-trajectory mixture training.

Experimental setup

Datasets: Dolci-Instruct-SFT Dolci-Think-SFT-32B

Benchmarks: GSM8K MATH-500 MBPP+ LiveCodeBench IFEval

Metrics: pass@1 (avg over 4 generations) exact match prompt/instruction accuracy (strict) average response length in tokens

Baselines: Linear Merge MergeMix TIES-Merge DARE-Ties SCE Math Expert Code Expert IF Expert

Builds on: MergeMix Linear merge / task arithmetic TIES-Merge DARE SCE qNEHVI

Closest papers by meaning

• MergeBench: A Benchmark for Merging Domain-Specialized LLMs (NeurIPS 2025 · similarity 0.90)
• Channel Merging: Preserving Specialization for Merged Experts (AAAI 2025 · similarity 0.89)
• OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging (ICLR 2026 · similarity 0.88)
• Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging (NeurIPS 2024 · similarity 0.88)
• Model Merging in Pre-training of Large Language Models (NeurIPS 2025 · similarity 0.88)
• Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent (ICML 2025 · similarity 0.88)