Research map / Large language models / Instruction and fine-tuning

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

ICML 2024 · Large language models › Instruction and fine-tuning · approach: Instruction data selection · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that simply selecting the 1,000 longest-response instructions from Alpaca-52k or Evol-Instruct-70k for SFT beats LIMA-1k and AlpaGasus-1k as judged by GPT-4 and PaLM-2, and that a GPT-3.5-refined version with NEFTune reaches 7.88% on AlpacaEval 2.0 with Llama-2-7B.

Problem

Instruction fine-tuning of LLMs is assumed to need high-quality curated data, but it is unclear which features make demonstrations high quality; existing selection methods (AlpaGasus, LIMA) rely on proprietary LLM scoring or costly manual curation.

Approach

Select the 1,000 training examples with the longest responses as a simple heuristic for IFT data, optionally refine them via GPT-3.5-Turbo introspection prompting, and combine with NEFTune noise; longer responses are argued to carry more learnable information and be harder to overfit.

Key results

• Alpaca-1k-longest achieves 46.3% average win rate vs LIMA-1k with only 28.3% losses under GPT-4 judging across five evaluation sets.
• Refined-Alpaca-1k-longest with NEFTune reaches 7.88% win rate on AlpacaEval 2.0 with Llama-2-7B, the second-best Llama-2-7B-based model on the leaderboard at the time.
• Refined-Alpaca-1k-longest with Mistral-7B-v0.1 reaches 11.74% AlpacaEval 2.0 win rate, versus 6.76% for LIMA-1k.

Measured effects reported in the paper

Surprising findings

• Most of the 1k longest Alpaca examples receive only about 3.5 quality score from GPT-3.5-Turbo, far lower than AlpaGasus examples, so length and LLM-judged quality barely overlap.
• Refined-Evol-Instruct-1k-longest underperforms refined Alpaca-1k-longest on AlpacaEval 2.0, which the authors attribute to GPT-3.5-Turbo's limited ability to refine very long demonstrations.

Limitations

• Length alone is not sufficient: LIMA training examples are about twice as long yet other factors matter, so the mechanism remains unclear.
• Evaluation relies largely on LLM judges (GPT-4, PaLM-2) with known length and position biases, though length-controlled and human checks are added.
• Refinement depends on GPT-3.5-Turbo, which is a proprietary model, and was less effective on very long Evol-Instruct examples.

Experimental setup

Datasets: Alpaca-52k Evol-Instruct-70k LIMA-1k AlpaGasus-1k Open-Hermes-1M

Benchmarks: AlpacaEval 2.0 MT-Bench Open LLM Leaderboard (ARC, MMLU, TruthfulQA, Winogrande) LIMA test set Vicuna test set Koala test set WizardLM test set Self-Instruct test set SCROLLS (GovReport, SummScreen, QMSum)

Metrics: win rate average response length MT-Bench score accuracy ROUGE

Baselines: AlpaGasus-1k LIMA-1k Alpaca-52k Evol-Instruct-70k Evol-Instruct-AlpaGasus-1k Llama-2-Chat-7B Tulu-2-DPO-7B Vicuna-7B

Builds on: LIMA AlpaGasus NEFTune Alpaca Evol-Instruct

Closest papers by meaning

• Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning (ICLR 2025 · similarity 0.91)
• PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (ICLR 2024 · similarity 0.91)
• What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (ICLR 2024 · similarity 0.91)
• The Best Instruction-Tuning Data are Those That Fit (NeurIPS 2025 · similarity 0.90)
• Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.90)
• Is In-Context Learning Sufficient for Instruction Following in LLMs? (ICLR 2025 · similarity 0.90)