Research map / Large language models / Instruction and fine-tuning
Evaluating the Zero-shot Robustness of Instruction-tuned Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows instruction-tuned LLMs (Flan-T5, Alpaca, T0++) degrade on 319 manually written, unobserved instruction phrasings across MMLU and Big-Bench Lite, and proposes soft-prompt tuning with a KL alignment loss over paraphrased instructions to partially recover robustness.
Problem
Instruction-tuned LLMs are supposed to generalize zero-shot to new tasks via natural language instructions, but it is unclear how sensitive they are to semantically equivalent rephrasings of instructions not seen during instruction tuning.
Approach
Collect 319 new instruction phrasings from NLP graduate students for MMLU and Big-Bench Lite tasks, measure the accuracy gap versus observed instructions, and align representations/output distributions of paraphrased instructions via a KL term while tuning only prefix soft-prompt embeddings.
Key results
- Using unobserved instructions reduces average accuracy by over five points across models (e.g., Alpaca-13B BBL-MC drops 14.0 points).
- Flan-T5-XL with PT+KL improves unobserved-instruction accuracy on BBL from 51.9 to 53.7 and on MMLU from 47.5 to 47.9, while PT alone gives smaller gains.
- Alpaca-7B with PT+KL improves unobserved BBL accuracy from 42.9 to 46.6 (+3.7) and MMLU unobserved from 39.7 to 41.8 (+2.1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use novel, unobserved rather than training-observed instructions | Flan-T5-3B · MMLU | accuracy: 48.1 -> 47.5 | Hurt |
| Use novel, unobserved rather than training-observed instructions | Alpaca-7B · MMLU | accuracy: 41.9 -> 39.7 | Hurt |
| Use novel, unobserved rather than training-observed instructions | Flan-T5-3B · BBL-BC | accuracy: 66.5 -> 61.1 | Hurt |
| Use novel, unobserved rather than training-observed instructions | Alpaca-7B · BBL-MC | accuracy: 32.1 -> 16.4 | Hurt |
| Use novel, unobserved rather than training-observed instructions | T0++ 11B · MMLU | accuracy: 48.3 -> 48.5 | No clear effect |
| Use soft prompt parameters and KL alignment loss (ablation) | Flan-T5-3B · MMLU, unobserved instructions | accuracy: +0.4 | Helped |
| Use soft prompt parameters and KL alignment loss (ablation) | Flan-T5-3B · BBL, unobserved instructions | accuracy: +1.8 | Helped |
| Use soft prompt parameters and KL alignment loss (ablation) | Alpaca-7B · MMLU, unobserved instructions | accuracy: +2.1 | Helped |
| Use soft prompt parameters and KL alignment loss (ablation) | Alpaca-7B · BBL, unobserved instructions | accuracy: +3.7 | Helped |
| Fine-tune all model parameters on paraphrases without KL alignment (ablation) | Flan-T5-3B · MMLU, unobserved instructions | accuracy: -7.4 | Hurt |
| Fine-tune all model parameters on paraphrases without KL alignment (ablation) | Flan-T5-3B · BBL, unobserved instructions | accuracy: -9.2 | Hurt |
| Fine-tune all model parameters and add KL alignment loss (ablation) | Alpaca-7B · MMLU, unobserved instructions | accuracy: +0.5 | Mixed |
Surprising findings
- Incorrect but observed instructions outperform appropriate but unobserved instructions on average in a closer-look experiment.
- Scaling Flan-T5 from 80M to 11B does not shrink the gap between observed and unobserved instruction performance.
Limitations
- Only mid-sized models (under 20B parameters) were evaluated, so results may not generalize to much larger models.
- Evaluation restricted to QA and classification task types.
- Robustness gains from the proposed method are modest and inconsistent across models and benchmarks.
Experimental setup
Datasets: MMLU Big-Bench Lite Flan collection Alpaca instruction set P3 GPT-4 paraphrased instructions
Benchmarks: MMLU Big-Bench Lite
Metrics: accuracy standard deviation across instructions average distance between representations
Baselines: Flan-T5-XL Alpaca-7B T0++ FT FT+KL PT
Builds on: Flan-T5 prefix tuning T0 Alpaca
Closest papers by meaning
- Instruction Tuning With Loss Over Instructions (NeurIPS 2024 · similarity 0.92)
- Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions (ICLR 2024 · similarity 0.91)
- Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following (AAAI 2024 · similarity 0.90)
- Enhancing NLU in Large Language Models Using Adversarial Noisy Instruction Tuning (AAAI 2025 · similarity 0.90)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.90)
- A Closer Look at the Limitations of Instruction Tuning (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.