Research map / Large language models / Instruction and fine-tuning
A Closer Look at the Limitations of Instruction Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows through token-distribution, human, and GPT-4 analyses that instruction tuning adds little knowledge, LoRA fine-tuning outperforms full fine-tuning and popular IT enhancement methods, and full fine-tuning increases hallucinations borrowed from similar IT instances.
Problem
Instruction tuning is widely used to turn base LLMs into chat models, but its limitations (knowledge acquisition, response quality, hallucination, and the benefit of improvement methods) remain underexplored.
Approach
Compares LoRA fine-tuning (LFT) and full-parameter fine-tuning (SFT) on open-source IT datasets via token-distribution shift analysis, human and GPT-4 evaluation, and causal analysis of hallucinations, finding that LFT mostly learns response initiation while SFT borrows tokens from similar IT instances; also proposes LIMA-Simple to reduce hallucination.
Key results
- Causal analysis: expert annotators found about 87% of hallucinated phrases from SFT models causally tied to IT instances, versus 13.9% for LFT.
- LFT models outperform AlpaGasus, WizardLM, and NEFTune variants on just-eval-instruct across metrics, per GPT-4 multi-aspect scoring.
- About 72% of human-marked hallucinated phrases exist verbatim in the IT dataset; LFT scaling from 1k to 326k samples yields no significant metric gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SFT instead of LFT | Instruction-tuned models · Randomly chosen hallucinated phrases | causal relation to IT dataset: 13.9% -> 87% | Hurt |
| Scale the LFT training dataset to 52× or 326× its original size | LFT models · just-eval-instruct | GPT-4 multi-aspect evaluation metrics | No clear effect |
| Use LFT on 1k samples instead of SFT on a dataset 326× larger | LLaMa-2 models · just-eval-instruct | factuality | Helped |
| Use LFT on open-domain data instead of domain-specific MedInstruct 52k SFT | LLaMa-2-7B · MedInstruct-test 216 | response quality | Helped |
| Simplify LIMA responses for SFT by removing extraneous information | LLaMa-2-7B SFT · just-eval-instruct | factuality | Helped |
| Simplify LIMA responses for SFT by removing extraneous information | LLaMa-2-7B SFT · just-eval-instruct | response depth | Hurt |
| Use SFT with AlpaGasus, WizardLM, or NEFTune instead of simple LFT | LLaMa-2 models · just-eval-instruct | response quality | Hurt |
| Use AlpaGasus, WizardLM, or NEFTune with LFT | LFT models · just-eval-instruct | response quality | Helped |
| Vary LoRA rank from the standard rank 8 to 2, 4, 16, or 32 (ablation) | LFT models · Instruction tuning | performance | No clear effect |
| Use SFT rather than LFT | Instruction-tuned models · just-eval-instruct | factuality | Hurt |
| Scale the IT dataset for LFT | LFT models · Knowledge-intensive domains including medicine | factuality | No clear effect |
Surprising findings
- Scaling the IT dataset by 52x to 326x does not significantly improve LoRA fine-tuned models on any of the five automatic metrics, including factuality.
- Responses from LoRA fine-tuned models on 1k samples outperform full fine-tuned models trained on 326x more data in factuality and helpfulness.
Limitations
- Limited to open-domain, single-turn, uni-modal text instruction tuning; does not cover domain/task-specific fine-tuning.
- Does not study DPO or RLHF alignment methods or retrieval-augmented generation.
- Relies on GPT-4 Turbo judge, so exact replication requires that specific model version.
Experimental setup
Datasets: Alpaca 52k MedInstruct 52k LIMA 1K databricks-dolly 15k Tulu-V2-Mix 326k just-eval-instruct 1k MedInstruct-test 216
Benchmarks: just-eval-instruct MedInstruct-test
Metrics: helpfulness clarity factuality depth engagement KL divergence base rank hallucination rate (human-annotated)
Baselines: AlpaGasus WizardLM NEFTune LLaMa-2-7B-chat
Builds on: LoRA NEFTune AlpaGasus WizardLM LLaMA-Factory
Closest papers by meaning
- On the Loss of Context Awareness in General Instruction Fine-tuning (NeurIPS 2025 · similarity 0.91)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.91)
- Instruction Tuning With Loss Over Instructions (NeurIPS 2024 · similarity 0.90)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.90)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.90)
- TAIA: Large Language Models are Out-of-Distribution Data Learners (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.