Research map / Large language models / Language, safety and interpretability
FLEX: Unifying Evaluation for Few-Shot NLP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add meta-training on FLEX meta-train data (ablation) | UniFew · FLEX zero-shot overall | accuracy: 56.5 -> 71.0 | Helped |
| add meta-training on FLEX meta-train data (ablation) | UniFew · FLEX few-shot overall | accuracy: 69.3 -> 77.9 | Helped |
| add meta-training (ablation) | UniFew · FLEX zero-shot class transfer | accuracy: 59.5 -> 75.6 | Helped |
| add meta-training (ablation) | UniFew · FLEX few-shot class transfer | accuracy: 75.8 -> 80.2 | Helped |
| add meta-training (ablation) | UniFew · FLEX zero-shot domain transfer | accuracy: 67.9 -> 87.6 | Helped |
| add meta-training (ablation) | UniFew · FLEX zero-shot task transfer | accuracy: 36.6 -> 41.1 | Helped |
| add meta-training (ablation) | UniFew · FLEX few-shot task transfer | accuracy: 54.3 -> 62.4 | Helped |
| provide few examples instead of zero | UniFew · FLEX overall | accuracy: 56.5 -> 69.3 | Helped |
| provide few examples instead of zero | UniFewmeta · FLEX overall | accuracy: 71.0 -> 77.9 | Helped |
| compare UniFewmeta with H-SMLMT | UniFewmeta · CNLL 4-shot | accuracy: 57.6 -> 79.7 | Helped |
| compare UniFewmeta with H-SMLMT | UniFewmeta · SciT 16-shot | accuracy: 80.4 -> 85.4 | Helped |
| add meta-training on SciT 4-shot (ablation) | UniFew · SciT 4-shot | accuracy: 65.1 -> 85.4 | Helped |
Closest papers by meaning
- Few-Shot Learning Evaluation in Natural Language Understanding (NeurIPS 2021 · similarity 0.91)
- The Surprising Effectiveness of Test-Time Training for Few-Shot Learning (ICML 2025 · similarity 0.89)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.89)
- Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning (AAAI 2024 · similarity 0.87)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.87)
- BatchPrompt: Accomplish more with less (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.