Research map / Large language models / Language, safety and interpretability
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train jointly across GLUE tasks rather than training separate task models | BiLSTM + attention + ELMo · GLUE test macro-average | macro-average: 66.5 -> 70.0 | Helped |
| add attention to the multi-task BiLSTM + ELMo (ablation) | BiLSTM + ELMo, multi-task · GLUE test macro-average | macro-average: 67.7 -> 70.0 | Helped |
| add ELMo to the multi-task BiLSTM + attention (ablation) | BiLSTM + attention, multi-task · GLUE test macro-average | macro-average: 65.6 -> 70.0 | Helped |
| add attention to the single-task BiLSTM + ELMo (ablation) | BiLSTM + ELMo, single-task · GLUE test macro-average | macro-average: 66.4 -> 66.5 | No clear effect |
| replace GloVe with ELMo embeddings (ablation) | BiLSTM, multi-task · GLUE test macro-average | macro-average: 64.2 -> 67.7 | Helped |
| replace GloVe with CoVe embeddings (ablation) | BiLSTM · GLUE test macro-average | macro-average: 63.9 -> 64.0; 64.2 -> 62.9 | Mixed |
| add attention to the multi-task BiLSTM (ablation) | BiLSTM, multi-task · GLUE test macro-average | macro-average: 64.2 -> 65.6 | Helped |
| add attention to the single-task BiLSTM (ablation) | BiLSTM, single-task · GLUE test macro-average | macro-average: 63.9 -> 63.9 | No clear effect |
Closest papers by meaning
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.89)
- Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.88)
- Linguini: A benchmark for language-agnostic linguistic reasoning (NeurIPS 2025 · similarity 0.88)
- SeqGPT: An Out-of-the-Box Large Language Model for Open Domain Sequence Understanding (AAAI 2024 · similarity 0.88)
- Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis (AAAI 2025 · similarity 0.88)
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.