Research map / Large language models / Instruction and fine-tuning
Revisiting Out-of-distribution Robustness in NLP: Benchmarks, Analysis, and LLMs Evaluations
Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, FangYuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add FreeLB adversarial training | vanilla fine-tuning · BOSS DS | OOD score: 47.38 -> 47.94 | Improved |
| add FreeLB adversarial training | vanilla fine-tuning · BOSS AN | OOD score: 36.19 -> 36.13 | No clear effect |
| add model ensemble | vanilla fine-tuning · BOSS AZ | OOD score: 90.94 -> 50.72 | Hurt |
| add EDA data augmentation | vanilla fine-tuning · BOSS MN/CN | OOD score: 89.40 -> 65.57 | Mixed |
| add focal loss | vanilla fine-tuning · BOSS CoNLL | OOD score: 69.10 -> 61.04 | Hurt |
| add label smoothing | vanilla fine-tuning · BOSS SE | OOD score: 49.90 -> 46.35 | Hurt |
| use in-context learning instead of zero-shot | T5-3B · BOSS SQuAD | OOD score: 46.64 -> 35.50 | Hurt |
| use in-context learning instead of zero-shot | Davinci3 · BOSS WNUT | OOD score: 32.54 -> 42.10 | Improved |
| use in-context learning instead of zero-shot | LLaMA-7B · BOSS CC | OOD score: 67.72 -> 89.70 | Improved |
| use in-context learning instead of zero-shot | Turbo · BOSS CoNLL | OOD score: 51.36 -> 57.02 | Improved |
| use OOD training-split examples (ICL*) instead of ID examples | Davinci3 · BOSS SE | OOD score: 68.88 -> 65.14 | Hurt |
| use OOD training-split examples (ICL*) instead of ID examples | Davinci3 · BOSS NQA | OOD score: 54.79 -> 56.15 | Improved |
Closest papers by meaning
- Brittlebench: Quantifying LLM robustness via prompt sensitivity (NeurIPS 2026 · similarity 0.88)
- Systematic Outliers in Large Language Models (ICLR 2025 · similarity 0.88)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.88)
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations (ICLR 2025 · similarity 0.88)
- Elo Uncovered: Robustness and Best Practices in Language Model Evaluation (NeurIPS 2024 · similarity 0.87)
- Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
