Research map / Large language models / Instruction and fine-tuning
Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Chain-of-Instructions (CoI), a dataset and tuning method where the output of one instruction feeds the next, built from Super-NaturalInstructions with GPT-3.5 Turbo composability checks, and shows CoI-tuned Alpaca-7B and Mistral-7B-Instruct generalize to unseen compositional, longer-chain, BBH, and multilingual summarization tasks.
Problem
LLMs are instruction-tuned mostly on single instructions and struggle to follow complex compositional instructions made of chained subtasks, where each subtask's output feeds the next.
Approach
Build chained compositional instruction data by using an LLM to shorten single instructions and check whether two (or more) tasks are composable, then generate the second-step output so models learn to solve subtasks step by step; fine-tune existing 7B instruction models on this CoI data.
Key results
- CoI_12-tuned Alpaca reaches 67.50 Rouge-L on CoI2-test vs 24.95 for base Alpaca, and CoI_123-tuned Mistral reaches 61.61 on CoI3-test vs 23.66 for base Mistral.
- On BIG-Bench Hard, CoI_123-tuned Mistral scores 29.39 Rouge-L vs 8.51 for base Mistral; CoI_12-tuned Alpaca scores 28.80 vs 14.36 base.
- LLM judge prefers CoI_123 Alpaca outputs over the CoI_1 baseline 81.04% of the time on CoI3-test.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add two-instruction training examples to single-instruction tuning | Mistral-7B-Instruct · CoI₂-test | Rouge-L: 39.72 -> 70.76 | Helped |
| Add two-instruction training examples to single-instruction tuning | Alpaca-7B · CoI₂-test | Rouge-L: 32.32 -> 67.50 | Helped |
| Add three-instruction training examples to single- and two-instruction tuning | Mistral-7B-Instruct · CoI₃-test | Rouge-L: 29.62 -> 61.61 | Helped |
| Add three-instruction training examples to single- and two-instruction tuning | Alpaca-7B · CoI₃-test | Rouge-L: 21.75 -> 67.49 | Helped |
| Add composed-instruction tuning | Mistral-7B-Instruct · BIG-Bench Hard | Rouge-L: 27.68 -> 29.39 | Helped |
| Add composed-instruction tuning | Alpaca-7B · BIG-Bench Hard | Rouge-L: 28.74 -> 28.80 | No clear effect |
| Use CoI₁₂ training | Mistral-7B-Instruct · WikiLingua-en-fr | Rouge-L (all): 8.03 -> 10.97 | Helped |
| Use CoI₁₂ training | Mistral-7B-Instruct · WikiLingua-es-en | Rouge-L (all): 11.22 -> 12.43 | Helped |
| Use CoI₁₂ training | Alpaca-7B · WikiLingua-es-en | Rouge-L (src): 2.47 -> 1.87 | Hurt |
| Use CoI₁₂ training | Alpaca-7B · WikiLingua-es-en | Rouge-L (tgt): 7.68 -> 7.13 | Hurt |
| Use concise composed instructions instead of the regular CoI instructions (ablation) | CoI₁₂-tuned Mistral-7B-Instruct · CoI₂-test | Rouge-L: 70.76 -> 59.19 | Hurt |
| Use concise composed instructions instead of the regular CoI instructions (ablation) | CoI₁₂₃-tuned Mistral-7B-Instruct · CoI₃-test | Rouge-L: 61.61 -> 68.97 | Helped |
Surprising findings
- Models fine-tuned only on CoI_3 data underperform on CoI_2 test and generate long hallucinated outputs, unlike CoI_12 mixtures.
- Irrelevant-CoI variants (mismatched chains) still achieve reasonable scores, suggesting gains partly come from format learning rather than true composition.
Limitations
- Composability checks by GPT-3.5 are noisy: only 75% of CoI2 and 59% of CoI3 samples were manually judged valid.
- Evaluation relies heavily on Rouge-L and an LLM judge; the Spanish-to-English downstream results are mixed for Alpaca.
- Only English-input Super-NaturalInstructions tasks are used, and the training set includes few Spanish tasks.
Experimental setup
Datasets: Super-NaturalInstructions WikiLingua BIG-Bench Hard
Benchmarks: CoI test sets (chain lengths 2-5) BIG-Bench Hard WikiLingua (en-fr, es-en)
Metrics: Rouge-L LLM preference win rate human preference valid output count
Baselines: Alpaca-7B base Mistral-7B-Instruct base chain-of-thought prompting least-to-most prompting CoI_1 single-instruction tuning
Builds on: Super-NaturalInstructions Chain-of-Thought prompting Least-to-Most prompting Alpaca Mistral-7B-Instruct
Closest papers by meaning
- Instruction Tuning With Loss Over Instructions (NeurIPS 2024 · similarity 0.90)
- #InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models (ICLR 2024 · similarity 0.89)
- Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.89)
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 · similarity 0.89)
- CoIN: A Benchmark of Continual Instruction Tuning for Multimodel Large Language Models (NeurIPS 2024 · similarity 0.89)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.