Research map / Large language models / Instruction and fine-tuning
AlpaGasus: Training a Better Alpaca with Fewer Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Filters Alpaca's 52k instruction data with ChatGPT accuracy scores to keep 9k high-quality samples, then fine-tunes LLaMA to get AlpaGasus, which beats full-data Alpaca on GPT-4-judged test sets at a fraction of the training time.
Problem
Widely used instruction-tuning datasets such as Alpaca's 52k contain many low-quality responses (incorrect or irrelevant) that can hurt instruction fine-tuning, and manual quality filtering is expensive.
Approach
Use a strong LLM (ChatGPT) as an automatic grader that scores each (instruction, input, response) triplet for accuracy on a 0-5 scale, then keep only triplets above a threshold (4.5) for standard Alpaca fine-tuning.
Key results
- AlpaGasus-9k outperforms Alpaca-52k on all four free-form test sets as judged by GPT-4, using only 17.75% of the data.
- AlpaGasus-13B achieves on average 90.1% of Text-Davinci-003's capacity, 81.2% of Claude, and 78.4% of ChatGPT.
- 7B training time drops from 80 minutes to 14 minutes (5.7x faster) on 4x A100 GPUs; 13B from 5.5h to 1h.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 7B · BBH | score: 33.01 -> 33.76 | Helped |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 7B · DROP | score: 25.87 -> 26.03 | Helped |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 7B · HumanEval | score: 11.69 -> 12.20 | Helped |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 7B · MMLU | score: 40.86 -> 38.78 | Hurt |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 13B · BBH | score: 38.67 -> 38.92 | Helped |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 13B · DROP | score: 33.84 -> 34.4 | Helped |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 13B · HumanEval | score: 15.74 -> 15.86 | Helped |
| Filter Alpaca data by response quality and train on 9k rather than all 52k examples | LLaMA 13B · MMLU | score: 47.89 -> 46.12 | Hurt |
| Use quality-filtered rather than randomly selected Alpaca training examples | LLaMA 7B (9k) · BBH | score: 31.89 -> 33.76 | Helped |
| Use quality-filtered rather than randomly selected Alpaca training examples | LLaMA 13B (9k) · BBH | score: 38.60 -> 38.92 | Helped |
| Filter Dolly training data by response quality instead of using all 15k examples | LLaMA 7B · HumanEval | score: 7.93 -> 9.78 | Helped |
| Use a 4.0 rather than 4.5 filtering threshold, yielding 39k examples (ablation) | Alpaca fine-tuning · Vicuna and Self-Instruct | GPT-4 evaluation | No clear effect |
Surprising findings
- Filtering on a coding-related subset removed 88.16% of samples versus 82.25% overall, leaving coding skills weaker than other skills.
- Alpaca-39k (threshold 4.0) did not beat Alpaca-52k on Vicuna and Self-Instruct, showing a loose filter retains harmful low-quality data.
Limitations
- Reliance on a proprietary LLM (ChatGPT) as the filter and GPT-4 as the judge, both closed-source.
- Filtering without skill balance hurts underrepresented categories such as coding.
- Human study sample is small (160 prompts, three annotators).
Experimental setup
Datasets: Alpaca 52k Databricks Dolly-15k GPT4LLM (Alpaca-GPT4) Self-Instruct test set Vicuna test set WizardLM test set Koala test set
Benchmarks: Vicuna test set Koala test set WizardLM test set Self-Instruct test set MMLU DROP HumanEval BBH
Metrics: win-tie-lose rate vs baseline GPT-4 score (1-10) human win/tie/lose benchmark accuracy training time and cost
Baselines: Alpaca-52k Alpaca-9k-random Text-Davinci-003 ChatGPT Claude-v1.1
Builds on: Alpaca LLaMA Vicuna evaluation prompt AlpacaEval
Closest papers by meaning
- Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning (ICML 2024 · similarity 0.90)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.89)
- Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning (ICLR 2025 · similarity 0.89)
- What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (ICLR 2024 · similarity 0.88)
- T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning (NeurIPS 2025 · similarity 0.88)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.