Research map / Large language models / Language, safety and interpretability
In-context learning analysis
Explain and extend in-context learning via latent-variable views, many-shot, and kNN prompting.
95 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2021–2026) · an approach in Language, safety and interpretability, keyword cluster “prompt · uncertainty · shot”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
36 papers in 2023–24 and 49 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 5.1% to 2.7%.
No longer reported by papers here: RTE (accuracy, last in 2024); QNLI (accuracy, last in 2024); MRPC (accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2021 · True Few-Shot Learning with Language Models (NeurIPS 2021 · 189 citations)
- 2022 · Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (NeurIPS 2022 · 79 citations)
- 2023 · Selective Annotation Makes Language Models Better Few-Shot Learners (ICLR 2023 · 62 citations)
- 2024 · ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context (AAAI 2024 · 13 citations)
- 2025 · Causal Prompting: Debiasing Large Language Model Prompting Based on Front-Door Adjustment (AAAI 2025 · 18 citations)
- 2026 · Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learning (AAAI 2026 · 2 citations)
Earliest papers here
- True Few-Shot Learning with Language Models (NeurIPS 2021 · 189 citations)
- Calibrate Before Use: Improving Few-shot Performance of Language Models (ICML 2021 · 72 citations)
- Few-Shot Learning Evaluation in Natural Language Understanding (NeurIPS 2021)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
SST-2 · accuracy (higher is better)
8 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 93.5 | Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners (ICLR 2022 · 74 citations) |
| 2023 | 95.8 | Fairness-guided Few-shot Prompting for Large Language Models (NeurIPS 2023 · 24 citations) |
| 2025 | 96.4 | Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective (ICML 2025) |
RTE · accuracy (higher is better)
6 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 65.3 | Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (NeurIPS 2022 · 79 citations) |
| 2023 | 69.7 | ADEPT: A DEbiasing PrompT Framework (AAAI 2023 · 28 citations) |
| 2024 | 84.1 | Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering (ICLR 2024 · 8 citations) |
QNLI · accuracy (higher is better)
5 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 73.3 | Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (NeurIPS 2022 · 79 citations) |
| 2023 | 85.41 | Compositional Exemplars for In-context Learning (ICML 2023 · 24 citations) |
| 2024 | 87.9 | Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering (ICLR 2024 · 8 citations) |
MR · accuracy (higher is better)
4 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 88.2 | Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners (ICLR 2022 · 74 citations) |
| 2024 | 88.58 | Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning (AAAI 2024 · 6 citations) |
MRPC · accuracy (higher is better)
4 papers, 2023–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 85 | ADEPT: A DEbiasing PrompT Framework (AAAI 2023 · 28 citations) |
What has been tried
67 papers here report 760 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 9%, hurt 21%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add ontology-based demonstration filtering | CoNLL2003, 1-shot · Micro-F1 | 71.12 -> 73.01 | Helped | ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context (AAAI 2024 · 13 citations) |
| Replace random prompt selection with adaptive routing | GYAFC · J | 38.3 -> 47.8 | Helped | Adaptive Prompt Routing for Arbitrary Text Style Transfer with Pre-trained Language Models (AAAI 2024 · 10 citations) |
| add word augmentation | VBench · Total Score | 80.37% -> 82.38% | Helped | The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation (CVPR 2025 · 7 citations) |
| fine-tune across 2–16 shots using random shot sampling | ImageNet-100, 16-shot · accuracy | 0.79 -> 0.75 | Hurt | Link-Context Learning for Multimodal LLMs (CVPR 2024 · 7 citations) |
| add multi-round dialogue alignment for prompt generation | SST-2 · accuracy | +3.12% | Helped | Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning (AAAI 2024 · 6 citations) |
| Use leave-one-out in-context tasks instead of direct input-output training | BIG-Bench Hard, 10-shot · accuracy | \num{51.5}\% -> \num{57.8}\% | Helped | The Surprising Effectiveness of Test-Time Training for Few-Shot Learning (ICML 2025 · 4 citations) |
| Add instruction rematching to instruction brainstorm | SuperNI validation · ROUGE-L (overall) | 36.27 -> 37.53 | Helped | MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction Following (ICLR 2024 · 4 citations) |
| Add single-intent division prompting | MIXSNIPS · Intent Acc. | 70.44 -> 72.90 | Helped | Divide-Solve-Combine: An Interpretable and Accurate Prompting Framework for Zero-shot Multi-Intent Detection (AAAI 2025 · 3 citations) |
| Repeat the query to match the instruction length instead of adding meaningful per-query instructions | FollowIR average · p-MRR | -3.1 -> -3.1 | No clear effect | Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models (ICLR 2025 · 3 citations) |
| Train with three fixed examples | MTEB, zero-shot · Avg. | 64.83 -> 61.50; decrease of 3.33 points | Hurt | Making Text Embedders Few-Shot Learners (ICLR 2025 · 3 citations) |
| Increase document relevance from random to clustering to nearest-neighbor linking | Wikipedia · perplexity | Helped | In-Context Pretraining: Language Modeling Beyond Document Boundaries (ICLR 2024 · 3 citations) | |
| Add LEAP low-level principles to few-shot CoT | DROP · accuracy | 72.05 -> 79.55 | Helped | In-Context Principle Learning from Mistakes (ICML 2024 · 3 citations) |
What reviewers will expect
Datasets and baselines named in the 32 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- GSM8K · 6 of 32 papers
- MATH · 4 of 32 papers
- SST-2 · 4 of 32 papers
- Subj · 3 of 32 papers
- TREC · 3 of 32 papers
- HotpotQA · 2 of 32 papers
- FEVER · 2 of 32 papers
- Financial PhraseBank · 2 of 32 papers
Baselines
- Zero-shot · 4 of 32 papers
- Random · 3 of 32 papers
- LoRA · 3 of 32 papers
- ICL · 2 of 32 papers
- BM25 · 2 of 32 papers
- Task-vector · 2 of 32 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 85 papers; from OpenAlex and the papers' affiliations).
- Google · 6 papers
- Google DeepMind · 5 papers
- Microsoft · 5 papers
- University of Virginia · 5 papers
- University of Science and Technology of China · 3 papers
- University of California, Berkeley · 3 papers
- Carnegie Mellon University · 3 papers
- Allen Institute for Artificial Intelligence · 3 papers
- Pennsylvania State University · 3 papers
- University of Illinois, Urbana Champaign · 2 papers
Use it
Download BibTeX of 35 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- Parallel and early-exit decoding (100 papers)
Elsewhere in the topic
- Uncertainty quantification for LLMs (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Safety alignment of LLMs (123 papers)
- Linguistic encoding in LLM representations (123 papers)
- Reasoning mechanisms and failure modes (114 papers)
- Speech and audio language models (111 papers)
- Sign and cross-lingual translation (96 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).