Research map / Large language models / Retrieval and knowledge
Inheriting Generalizable Knowledge from LLMs to Diverse Vertical Tasks
Chang Liu, boyu shi, Xu Yang, Qiufeng Wang, Xin Geng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add spectral alignment (ablation) | MASA 877M · DollyEval | Rouge-L: 23.38 -> 24.46 | Improved |
| add spectral alignment (ablation) | MASA 877M · S-NI | Rouge-L: 17.35 -> 18.51 | Improved |
| add adaptive scaling (ablation) | MASA 877M · DollyEval | Rouge-L: 23.58 -> 24.46 | Improved |
| add adaptive scaling (ablation) | MASA 877M · S-NI | Rouge-L: 17.20 -> 18.51 | Improved |
| add adaptive scaling (ablation) | MASA 877M · BoolQ | accuracy: 72.14 -> 73.36 | Improved |
| increase gene matrix ratio from 3% to 23% (ablation) | MASA 877M · avg of 6 tasks | average score: 45.52 -> 46.91 | Improved |
| increase gene matrix ratio from 23% to 40% (ablation) | MASA 877M · avg of 6 tasks | average score: 46.91 -> 45.28 | Hurt |
| align gene matrices with MASA-OLMo initialization instead of random init | 16L-877M · Dialogue avg Rouge-L | Rouge-L: 17.53 -> 18.42 | Improved |
| initialize with MASA-OLMo instead of distillation | 12L-709M · Dialogue avg Rouge-L | Rouge-L: 16.88 -> 18.71 | Improved |
| initialize with MASA-DeepSeek instead of Scratch | 12L-267M · Language understanding avg | average accuracy: 52.53 -> 54.40 | Improved |
| use EEP expert pruning instead of Scratch | 12L-267M · Language understanding avg | average accuracy: 52.53 -> 41.85 | Hurt |
| use knowledge distillation instead of Scratch | 16L-877M · Dialogue avg Rouge-L | Rouge-L: 17.53 -> 17.06 | No clear effect |
Closest papers by meaning
- Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective (ICLR 2024 · similarity 0.88)
- A transfer learning framework for weak to strong generalization (ICLR 2025 · similarity 0.88)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.88)
- The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning (ICLR 2024 · similarity 0.87)
- Knowledge Fusion of Large Language Models (ICLR 2024 · similarity 0.87)
- Quantifying Generalization Complexity for Large Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
