Research map / Large language models / Instruction and fine-tuning
Enhancing NLU in Large Language Models Using Adversarial Noisy Instruction Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adversarial Noisy Instruction Tuning (ANIT), which trains LLMs with controlled noisy instructions and a two-step noise-response adversarial regularizer scaled by a semantic-distortion score, improving NLU tasks with LoRA fine-tuning.
Problem
Instruction tuning for LLMs yields weaker gains on NLU tasks, which need high-precision outputs, and high-quality task instruction data is scarce and costly to build. Classic adversarial training is also too expensive for LLMs.
Approach
Construct noisy variants of task instructions (random, in-context, and opposite noise), quantify their semantic distortion with Levenshtein-based rule evaluation or embedding-based LM evaluation, and use that score to scale a cheap two-step adversarial perturbation (Noise Response Method) added as a regularization loss during LoRA instruction tuning.
Key results
- ANIT with random replacement noise improves LLaMA3-8B on detailed instructions, e.g., ConLL03 F1 from 92.71 to 93.66 and AGNews from 95.02 to 95.89 (Table 1).
- LM-based semantic distortion evaluation gives larger gains than rule-based, e.g., LLaMA2-7B 14Rest +1.96 vs +1.84 and LLaMA3-8B AGNews +2.01 vs +1.37 (Table 3 comparison).
- In few-shot cross-dataset transfer with 5% data, LLaMA2-7B on ConLL03 rises from 86.98 to 89.26 F1 (Table 4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ANIT using rule-based semantic distortion evaluation instead of standard instruction tuning | LLaMA3-8B · NLU tasks (CoNLL03, Ontonotes, NYT, SciERC, SST2, AGNews, 14Lap, 14Rest) | Micro-F1: 92.71 -> 93.66; 91.53 -> 92.42; 91.57 -> 92.94; 48.84 -> 50.57; 97.40 -> 97.81; 95.02 -> 95.69; 64.37 -> 66.13; 73.84 -> | Helped |
| Use LM-based rather than rule-based semantic distortion evaluation in ANIT (ablation) | ANIT on LLaMA3-8B · NLU tasks (CoNLL03, Ontonotes, NYT, SciERC, SST2, AGNews, 14Lap, 14Rest) | Micro-F1: 93.66 -> 93.79; 92.42 -> 92.82; 92.94 -> 93.58; 50.57 -> 50.66; 97.81 -> 98.16; 95.69 -> 95.87; 66.13 -> 66.16; 74.93 -> | Helped |
| Apply ANIT using LM-based semantic distortion evaluation instead of standard instruction tuning | LLaMA2-7B · NLU tasks (CoNLL03, Ontonotes, NYT, SciERC, SST2, AGNews, 14Lap, 14Rest) | Micro-F1: 92.63 -> 93.94; 90.39 -> 92.03; 90.89 -> 92.92; 42.18 -> 45.53; 96.80 -> 97.49; 94.08 -> 96.02; 62.97 -> 64.69; 72.37 -> | Helped |
| Use random replacement noise in ANIT rather than standard instruction tuning | Gemma-2B, LLaMA2-7B, and LLaMA3-8B · NLU tasks | Micro-F1: +0.99 to +3.18 | Helped |
| Increase noisy-instruction strength factor ω above 0.3 (ablation) | ANIT · NLU tasks | Micro-F1 | Hurt |
| Use ANIT with noise strength factor ω from 0.1 to 0.5 (ablation) | ANIT · NLU tasks | Micro-F1 | Helped |
| Use ANIT instead of standard training in cross-dataset few-shot transfer (ablation) | LLaMA2-7B trained on CoNLL2003 and evaluated with Ontonotes samples · CoNLL03 and Ontonotes | Micro-F1: 5%: 86.98 -> 89.26; 78.69 -> 81.43 | Helped |
| Use ANIT rather than standard instruction tuning for NLU tasks | LLMs · Detailed and concise instructions across NLU tasks | Micro-F1 | Mixed |
Surprising findings
- Some noise types degrade performance on specific tasks with detailed instructions, which the authors attribute to pre-training strategies and instruction construction.
- Opposite-style noise in some settings reduces performance on individual datasets despite overall gains.
Limitations
- Gains on some tasks and noise types are small or negative, especially with detailed instructions.
- Relies on hand-designed noise types and hyperparameters (noise strength factor, global noise factor) without a broader study.
- LM-based evaluation depends on a proprietary embedding API (text-embedding-3-large).
Experimental setup
Datasets: CoNLL03 OntoNotes NYT SciERC SST2 AGNews 14Lap 14Rest Alpaca instructions
Benchmarks: CoNLL03 OntoNotes NYT SciERC SST2 AGNews 14Lap 14Rest
Metrics: Micro-F1
Baselines: FreeLB PGD FGSM FGM standard instruction tuning (no noise)
Builds on: FreeLB FGM PGD LoRA
Closest papers by meaning
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.90)
- Instruction Tuning With Loss Over Instructions (NeurIPS 2024 · similarity 0.90)
- Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions (ICLR 2024 · similarity 0.89)
- Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis (AAAI 2025 · similarity 0.89)
- A Closer Look at the Limitations of Instruction Tuning (ICML 2024 · similarity 0.88)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.