Research map / Large language models / Language, safety and interpretability
Transport and Merge: Cross-Architecture Merging for Large Language Models
Chenhang Cui, Binyun Yang, Fei Shen, Yuxin Chen, Jingnan Zheng, Xiang Wang, An Zhang, Tat-Seng Chua
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply OT-guided fusion with adaptation | Malaysian-LLaMA-3-1B-Instruct · MalayMMLU (Humanities) | accuracy (%): 42.30 -> 48.81 | Improved |
| apply OT-guided fusion with adaptation | Malaysian-LLaMA-3-1B-Instruct · MalayMMLU (Social Science) | accuracy (%): 40.92 -> 46.91 | Improved |
| add post-fusion residual-frozen adaptation (ablation) | Fused model · MalayMMLU (Humanities) | accuracy (%): 46.19 -> 48.81 | Improved |
| add post-fusion adaptation (ablation) | Fused model · ARC (Indonesian) | accuracy (%): 24.0 -> 23.7 | Hurt |
| add post-fusion adaptation (ablation) | Fused model · Indonesian benchmarks | average accuracy (%): 39.18 -> 39.30 | Improved |
| apply fusion without adaptation | LLaMA-3-1B-Indonesian-QLoRA · Indonesian benchmarks | average accuracy (%): 38.43 -> 39.18 | Improved |
| apply fusion with adaptation | LLaMA-3-1B-Instruct · CMMLU Cantonese | average accuracy (%): 25.26 -> 27.44 | Improved |
| apply fusion without adaptation | Typhoon2-1B-Instruct · MGSM (Thai) | score: 0.50 -> 0.64 | Improved |
| apply fusion without adaptation | Typhoon2-1B-Instruct · MMLU (Thai) | score: 0.15 -> 0.13 | Hurt |
| add adaptation after fusion (ablation) | Fused model · Thai benchmarks | average score: 0.45 -> 0.50 | Improved |
| fuse general model into finance-specialized target without adaptation | LLaMA-3-1B-TEL-A-finance · finance MMLU subsets | average score: 0.32 -> 0.34 | Improved |
| apply fusion with adaptation | medical LLaMA-3-1B · medical MMLU subsets | average accuracy (%): 46.30 -> 47.87 | Improved |
Closest papers by meaning
- Model Merging in Pre-training of Large Language Models (NeurIPS 2025 · similarity 0.89)
- Channel Merging: Preserving Specialization for Merged Experts (AAAI 2025 · similarity 0.89)
- MergeBench: A Benchmark for Merging Domain-Specialized LLMs (NeurIPS 2025 · similarity 0.88)
- Activation-Informed Merging of Large Language Models (NeurIPS 2025 · similarity 0.88)
- MergeNet: Knowledge Migration Across Heterogeneous Models, Tasks, and Modalities (AAAI 2025 · similarity 0.88)
- AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models with Unsupervised Coefficient Optimization (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
