Research map / Large language models / Instruction and fine-tuning
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes dynamic logit fusion that transfers knowledge from a series of task-specific small models (1.1B/7B LLaMA2 experts) to a 13B LLaMA2 at each decoding step, with per-step weights learned via a KL-constrained objective, requiring no gradient updates to the large model.
Problem
Logit arithmetic for weak-to-strong transfer uses a static, pre-set weight on the small model's fine-tuning delta and a single expert, which limits performance and generalization to unseen tasks; fine-tuning large models is memory-expensive.
Approach
At each decoding step, per-expert weights alpha_t are chosen by minimizing a symmetric KL-distance matching objective between the fused large-model distribution and each small expert's shift from its base, so knowledge from multiple task experts is adaptively fused into a large model without training it.
Key results
- Transferring from the 7B expert to 13B closes 96.4% of the performance gap to 13B full fine-tuning in single-task settings (46.09 avg vs 44.13 expert-level baseline shown in table; 13B full FT 47.79).
- In multi-task settings, transfer from 7B reaches 39.42 average vs 34.89 for 7B multi-task tuning and 45.68 for 13B multi-task tuning, closing 86.3% of the gap.
- On unseen MMLU in the multi-task setting, the method reaches 51.31 vs 50.58 for 13B multi-task tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dynamic logit fusion instead of Proxy Tuning | LLaMA2-13B with 7B experts · GSM8K | EM: 37.68 -> 39.34 | Helped |
| Use dynamic logit fusion instead of Proxy Tuning | LLaMA2-13B with 7B experts · TruthfulQA | accuracy: 61.02 -> 61.56 | Helped |
| Use dynamic logit fusion instead of Proxy Tuning | LLaMA2-13B with 7B experts · TriviaQA | EM: 52.81 -> 57.11 | Helped |
| Use dynamic logit fusion instead of Proxy Tuning | LLaMA2-13B with 7B experts · CNN/DM | ROUGE-2: 14.37 -> 15.31 | Helped |
| Use dynamic logit fusion instead of Proxy Tuning | LLaMA2-13B with 7B experts · MMLU | accuracy: 56.24 -> 57.15 | Helped |
| Transfer knowledge from 7B experts with dynamic fusion rather than full fine-tuning the 13B model | LLaMA2-13B · TriviaQA | EM: 56.36 -> 57.11 | Helped |
| Transfer knowledge from 7B experts rather than full fine-tuning the 13B model | LLaMA2-13B · CNN/DM | ROUGE-2: 15.50 -> 15.31 | Hurt |
| Transfer knowledge from 7B experts with dynamic fusion rather than 7B multi-task tuning | LLaMA2-13B · Multi-task average | average score: 34.89 -> 39.42 | Helped |
| Transfer knowledge from 7B experts with dynamic fusion rather than 7B multi-task tuning | LLaMA2-13B · MMLU (unseen task) | accuracy: 35.82 -> 51.31 | Helped |
| Transfer knowledge from 7B experts with dynamic fusion rather than 7B multi-task tuning | LLaMA2-13B · Seen tasks average | average score: 33.96 -> 27.53 | Hurt |
| Replace predefined alpha values with dynamically adjusted alpha (ablation) | Logit transfer method · Single-task and multi-task benchmarks | performance | Helped |
| Add in-context learning to dynamic logit fusion | LLaMA2-13B with 7B experts · Multi-domain tasks | performance: 18.3% | Helped |
Surprising findings
- The method outperforms 13B full fine-tuning on TriviaQA in the single-task setting.
- On the multi-task setting, the method underperforms multi-task tuning on some seen tasks such as TriviaQA.
Limitations
- Multiple models in the decoding loop slow inference (about 2.5x slower than 13B alone).
- Only applicable to generative models sharing a vocabulary; not tested on discriminative models.
- Cannot access expert training data, causing bottleneck on task conflicts in multi-task settings.
Experimental setup
Datasets: GSM8K TruthfulQA TriviaQA CNN/DailyMail MMLU
Benchmarks: GSM8K TruthfulQA TriviaQA CNN/DM MMLU
Metrics: exact match accuracy Rouge-2 average performance performance gap closed
Baselines: Proxy Tuning Full Fine-tuning (13B) LoRA Tuning (13B) Multi-Task Tuning In-Context Learning (5-shot) Task Arithmetic
Builds on: Proxy Tuning DPO Contrastive Decoding Task Arithmetic
Closest papers by meaning
- A transfer learning framework for weak to strong generalization (ICLR 2025 · similarity 0.88)
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection (AAAI 2025 · similarity 0.87)
- Model Fusion through Bayesian Optimization in Language Model Fine-Tuning (NeurIPS 2024 · similarity 0.87)
- Knowledge Fusion of Large Language Models (ICLR 2024 · similarity 0.87)
- Quantifying the Gain in Weak-to-Strong Generalization (NeurIPS 2024 · similarity 0.87)
- Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.