Research map / Large language models / Instruction and fine-tuning
From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
Hao Chen, Qi Zhang, Liyao Li, Zhanming Shen, Wentao Ye, Lirong Gao, NINGTAO WANG, Xing Fu, Xiaoyu Shen, Junbo Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add P2D parameter-guided data selection instead of Data Whisperer | sparse head adaptation · Average over 3 datasets x 3 models | average score: 49.38 -> 51.87 | Improved |
| add P2D sparse head adaptation (10% heads) instead of LoRA on full data | full data fine-tuning · Average over 3 datasets x 3 models | average score: 54.25 -> 56.22 | Improved |
| update 10% of heads chosen by P2D instead of random heads (ablation) | full data fine-tuning · Average over 3 datasets x 3 models | average score: 46.64 -> 56.22 | Improved |
| add P2D sparse head adaptation to P2D data selection versus full parameters (ablation) | P2D data selection · Average over 3 datasets x 3 models | average score: 50.58 -> 51.87 | Improved |
| reduce data ratio from 100% to 10% (GSM8K, Qwen-2.5-7B) (ablation) | P2D · GSM8K | EM: 83.03 -> 82.56 | No clear effect |
| reduce data ratio from 100% to 10% (BioInstruct, Qwen-2.5-7B) (ablation) | P2D · BioInstruct | ROUGE-L: 41.92 -> 32.78 | Hurt |
| reduce data ratio from 100% to 10% (DialogSum, Llama-3-8B) (ablation) | P2D · DialogSum | ROUGE-L: 48.15 -> 42.45 | Hurt |
| reduce head ratio from 100% to 10% (GSM8K, Qwen-2.5-7B) (ablation) | P2D · GSM8K | EM: 80.05 -> 82.56 | Improved |
| reduce head ratio from 100% to 10% (BioInstruct, Qwen-3-8B) (ablation) | P2D · BioInstruct | ROUGE-L: 33.45 -> 32.89 | Hurt |
| scale model from 1.5B to 32B with P2D sparse adaptation versus Full SFT (GSM8K) | P2D-sparse head adaptation · GSM8K (full data) | EM: 63.45 -> 65.23 at 1.5B; 88.94 -> 90.23 at 32B | Improved |
| add P2D sparse adaptation versus Full SFT on general benchmarks after UltraChat | Qwen-2.5-7B · MMLU | accuracy: 56.34 -> 67.45 | Improved |
| use W1 head scoring instead of cosine similarity (ablation) | P2D · GSM8K | EM: 63.28 -> 78.82 | Improved |
Closest papers by meaning
- Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples (ICML 2025 · similarity 0.91)
- Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models (NeurIPS 2025 · similarity 0.90)
- Alignment Dynamics in LLM Fine-Tuning (NeurIPS 2026 · similarity 0.90)
- L3Ms — Lagrange Large Language Models (ICLR 2025 · similarity 0.89)
- Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing (ICLR 2025 · similarity 0.89)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
