Research map / Large language models / Instruction and fine-tuning
Beyond IID: Optimizing Instruction Finetuning from the Perspective of Instruction Interaction and Dependency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Investigates interaction and dependency patterns between instruction categories via causal interventions, then optimizes SFT data category proportions with a linear program (EE-CPO) and orders training with a dependency-taxonomy curriculum (DT-CSFT).
Problem
Instruction selection for SFT of LLMs typically scores individual instructions by quality and ignores joint interactions and dependencies between instruction categories, leading to suboptimal instruction sets and training schedules.
Approach
Measure pairwise category correlations (effect equivalence coefficients) and category dependencies (via ablation with Wilcoxon tests on PPL), then reweight category proportions with a linear program and reorder training so preliminary categories (math, coding) come first, going beyond the iid assumption.
Key results
- EE-CPO (50k) reaches MT-Bench 7.17 on Qwen1.5 and 7.51 on Llama3, versus 7.02 and 7.13 for DEITA (50k).
- EE-CPO (50k) reaches AlpacaEval 2.0 of 11.29 on Qwen1.5 and 11.47 on Llama3, versus 10.41 and 9.74 for DEITA (50k).
- Correlation and dependency patterns induced from Qwen-1.5-7B transfer to improve Llama3-8B, per the authors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 10k | Qwen-1.5-7B · MT-Bench | score: 7.08 -> 7.09 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 10k | Llama3-8B · MT-Bench | score: 7.00 -> 7.17 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 10k | Qwen-1.5-7B · AlpacaEval 2.0 | score: 8.98 -> 9.94 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 10k | Llama3-8B · AlpacaEval 2.0 | score: 7.84 -> 8.09 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 50k | Qwen-1.5-7B · MT-Bench | score: 7.02 -> 7.17 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 50k | Llama3-8B · MT-Bench | score: 7.13 -> 7.51 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 50k | Qwen-1.5-7B · AlpacaEval 2.0 | score: 10.41 -> 11.29 | Helped |
| Optimize instruction-category proportions with EE-CPO instead of DEITA at 50k | Llama3-8B · AlpacaEval 2.0 | score: 9.74 -> 11.47 | Helped |
| Arrange category learning in dependency-taxonomy order, emphasizing preliminary categories early | DT-CSFT · MT-Bench and AlpacaEval 2.0 | benchmark performance | Helped |
| Add preliminary-category instructions throughout epochs with Mix+ instead of arranging them early | Mix+ · MT-Bench and AlpacaEval 2.0 | benchmark performance | No clear effect |
Surprising findings
- Adding more instructions does not necessarily help: DT-CSFT outperforms the Mix+ baseline that adds extra preliminary-category data.
- Many category pairs show antagonistic (negative) effect coefficients, so adding one category can degrade performance on another.
Limitations
- Correlation analysis is done on 7B-scale models and assumed to scale to larger models without verification.
- Category importance weights rely on the proportions from DEITA-selected sets, and the pairwise effect measure depends on the choice of base model and category set.
Experimental setup
Datasets: Alpaca GPT4 LIMA UltraChat WildChat MetaMath Code Alpaca 20K COIG-CQIA LogiQA ShareGPT-Chinese-English-90k BELLE
Benchmarks: MT-Bench AlpacaEval 2.0
Metrics: MT-Bench score AlpacaEval 2.0 win rate
Baselines: Random Selection Instag IFD DEITA Mix+
Builds on: DEITA Instag IFD curriculum learning
Closest papers by meaning
- #InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models (ICLR 2024 · similarity 0.88)
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 · similarity 0.88)
- DINGO: Towards Diverse and Fine-Grained Instruction-Following Evaluation (AAAI 2024 · similarity 0.88)
- XIFBench: Evaluating Large Language Models on Multilingual Instruction Following (NeurIPS 2025 · similarity 0.87)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.87)
- SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.