Research map / Architectures and efficiency / Pruning and efficient tuning
Navigating the Accuracy-Size Trade-Off with Flexible Model Merging
Akash Dhasade, Divyansh Jhunjhunwala, Milos Vujasinovic, Gauri Joshi, Anne-Marie Kermarrec
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| double deployed size from 1x to 2x with FlexMerge | TA on 8 vision tasks (ViT-B/32) · 8 vision tasks | average accuracy: 67.5 -> >80 | Improved |
| grow merged size from ~3x to ~6x | Consensus on 30 vision tasks (ViT-B/32) · 30 vision tasks | average accuracy: 76 -> 84.5 | Improved |
| grow merged size from 1x to 3x | TA on 11 PEFT models (T0-3B) · 11 PEFT tasks | average accuracy: 59 -> 66.2 | Improved |
| grow merged size from 1x to 2x | TA on 7 FFT models (T5-Large) · 7 FFT tasks | average accuracy: +9% to 75 | Improved |
| grow merged size from 2.2x to 4x | EMR-Merging on 7 FFT models (T5-Large) · 7 FFT tasks | average accuracy: 85.2 -> >86 | Improved |
| block-wise greedy pairwise merging instead of Channel Merging K-Means | TA (ViT-B/32) · 8 vision tasks, integer sizes | average accuracy | Improved |
| use min similarity instead of max/average between groups (ablation) | FlexMerge · 8 vision tasks (ViT-B/32) | average accuracy | Improved |
| greedy merging order instead of left-to-right or right-to-left (ablation) | FlexMerge · 8 vision tasks (ViT-B/32) | average accuracy | Improved |
| cosine-similarity block selection instead of random selection (ablation) | FlexMerge · 8 vision tasks (ViT-B/32) | average accuracy | Improved |
| increase size beyond 1x for ranking of algorithms | vanilla averaging vs TIES · vision tasks | average accuracy | Mixed |
| larger merged size on inference latency | FlexMerge vs standard merging · ViT-B/32 inference | inference time: 12.30 ms -> 12.21 ms | No clear effect |
| larger merged size on inference latency | FlexMerge vs standard merging · ViT-L/14 inference | inference time: 118.70 ms -> 120.53 ms | No clear effect |
Closest papers by meaning
- MergeBench: A Benchmark for Merging Domain-Specialized LLMs (NeurIPS 2025 · similarity 0.91)
- Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging (NeurIPS 2024 · similarity 0.90)
- OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging (ICLR 2026 · similarity 0.90)
- Dynamic Model Merging Made Slim (NeurIPS 2026 · similarity 0.90)
- EMR-Merging: Tuning-Free High-Performance Model Merging (NeurIPS 2024 · similarity 0.89)
- Merging on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
