Research map / Large language models / Reasoning and chain of thought

ProFuser: Progressive Fusion of Large Language Models

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ProFuser, which fuses heterogeneous LLMs by scoring source-model advantage in both inference mode (reward-model voting) and training mode (min cross-entropy), progressively moving from inference-mode to training-mode fusion.

Problem

Existing LLM fusion methods select advantageous source models using only ground-truth cross-entropy under teacher forcing, which poorly reflects generation quality at inference time.

Approach

Evaluate source models in both training mode (min-CE on GPT-4 ground truth) and inference mode (reward-model voting over generated responses), then fuse progressively: first inference-mode signals, then training-mode signals with a weighted combination of SFT and KL token-level losses.

Key results

• Vicuna-7B-v1.5-ProFuser reaches 55.11 average across six benchmarks vs 53.46 for the Vicuna-7B-v1.5 base (+3.09%) and 54.20 for continual SFT.
• ProFuser outperforms FuseLLM-style Vicuna-7B-v1.5-Fuse (54.53 average) and ReverseFuse (54.30) and SimulFuse (54.56) on average.
• Under the knowledge/reasoning/safety grouping, ProFuser attains 53.48 average vs 51.82 for Vicuna-7B-v1.5.

Surprising findings

• Reverse ordering (training mode first, then inference mode) underperforms FuseLLM and degrades overall performance.
• Fusing with the weaker MPT-7B-8K-Chat gives larger gains on HellaSwag and WinoGrande than fusing with the stronger Llama-2-7B-Chat.

Limitations

• Gains on GSM8K are modest and slightly below FuseLLM, attributed to noisy inference outputs in math tasks.
• Evaluation limited to three 7B source models and one target model.

Experimental setup

Datasets: Orca-Best (OpenOrca GPT-4 subset)

Benchmarks: MMLU HellaSwag ARC-Challenge Winogrande GSM8K TruthfulQA

Metrics: accuracy average score

Baselines: FuseLLM SimulFuse ReverseFuse Continual SFT SLERP TIES DARE RMFuser GTLenFuser

Builds on: FuseLLM knowledge distillation Task Arithmetic Ties-Merging

Closest papers by meaning

• Knowledge Fusion of Large Language Models (ICLR 2024 · similarity 0.89)
• A Probabilistic Perspective on Unlearning and Alignment for Large Language Models (ICLR 2025 · similarity 0.88)
• Model Merging in Pre-training of Large Language Models (NeurIPS 2025 · similarity 0.88)
• PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs (ICLR 2025 · similarity 0.88)
• metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.87)
• Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.87)