Research map / Architectures and efficiency / Neural architecture search

HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models

NeurIPS 2025 · Architectures and efficiency › Neural architecture search · approach: AutoML pipeline search · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HM3, a hierarchical multi-objective model merging framework that combines DARE+TIES merging in parameter space with PPO-based search over inference paths across layers of fine-tuned Llama-2-7B variants in architecture space, yielding a Pareto front of merged models.

Problem

Model merging is largely limited to identical architectures in parameter space; merging across architectures (layer-level) has a vast, hard search space, and existing methods return a single merged model rather than trade-offs across user-preferred tasks.

Approach

Model architecture-space merging as an MDP in which a token traverses layers across fine-tuned and parameter-merged models; PPO learns policy and value networks to optimize the inference path, with a weight-vector-based multi-objective scheme (MOEA/D-style simplex sampling) that produces a Pareto set of merged models.

Key results

• On the three-task setting, HM3 scores 44.68 translation, 45.62 math, and 43.62 code, versus best baseline DARE+TIES at 33.93, 38.20, 28.20.
• HM3 reports hypervolume 1.6824 versus 1.4564 for HM3 without architecture-space optimization and 1.2486 without parameter-space optimization.
• On ViT-B/32 vision merging, HM3 averages 66.91% vs 64.99% for Dare Ties; on ViT-L/14 it averages 80.30% vs 79.26%.

Measured effects reported in the paper

Surprising findings

• HM3 without parameter-space optimization performs worse than HM3 without architecture-space optimization on the single-objective tasks, and the RL reward only begins to rise after roughly 200 episodes.

Limitations

• Experiments limited to three 7B Llama-2-derived models and a single sampled weight vector for the main table.
• Architecture search restricted to inference-path reordering rather than full architecture search, which the authors note may reduce merged-model performance.
• Code release deferred until after review; no compute or cost details reported.

Experimental setup

Datasets: WMT14 WMT16 IWSLT2017 GSM8K HumanEval DTD GTSRB RESISC45 SUN397 SVHN

Benchmarks: WMT14 WMT16 IWSLT2017 GSM8K HumanEval DTD GTSRB RESISC45 SUN397 SVHN

Metrics: chrF flexible match accuracy pass@100 hypervolume accuracy

Baselines: Task Arithmetic TIES DARE+TIES HM3 w.o. para. opt. HM3 w.o. archi. opt.

Builds on: DARE TIES-Merging Task Arithmetic PPO MOEA/D

Closest papers by meaning

• Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging (ICML 2025 · similarity 0.87)
• Mixture-Trained Merging for Unified Multi-Objective Models (NeurIPS 2026 · similarity 0.87)
• Merging on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging (NeurIPS 2025 · similarity 0.86)
• Whoever Started the interference Should End It: Guiding Data-Free Model Merging via Task Vectors (ICML 2025 · similarity 0.86)
• Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent (ICML 2025 · similarity 0.86)
• CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging (ICML 2025 · similarity 0.85)