Research map / Large language models / Language, safety and interpretability
Data and evaluation methodology
Assess datasets, evaluation failures, and summarization with description-length and data-curation tools.
136 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Language, safety and interpretability, keyword cluster “prompt · uncertainty · shot”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
39 papers in 2023–24 and 65 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 5.6% to 3.5%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Extracting Topical Phrases from Clinical Documents (AAAI 2016 · 20 citations)
- 2017 · Prerequisite Skills for Reading Comprehension: Multi-Perspective Analysis of MCTest Datasets and Systems (AAAI 2017 · 18 citations)
- 2018 · Dialog-based Interactive Image Retrieval (NeurIPS 2018 · 117 citations)
- 2019 · Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices (NeurIPS 2019 · 41 citations)
- 2020 · TabFact: A Large-scale Dataset for Table-based Fact Verification (ICLR 2020 · 235 citations)
- 2021 · CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review (NeurIPS 2021 · 96 citations)
- 2022 · Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (NeurIPS 2022 · 43 citations)
- 2023 · Solving Math Word Problems concerning Systems of Equations with GPT-3 (AAAI 2023 · 32 citations)
- 2024 · Bootstrapping Large Language Models for Radiology Report Generation (AAAI 2024 · 72 citations)
- 2025 · FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models (CVPR 2025 · 8 citations)
- 2026 · PPGPT: Transferring Next-Token Modeling from Language to PPG Signals (AAAI 2026 · 1 citation)
Earliest papers here
- Extracting Topical Phrases from Clinical Documents (AAAI 2016 · 20 citations)
- Prerequisite Skills for Reading Comprehension: Multi-Perspective Analysis of MCTest Datasets and Systems (AAAI 2017 · 18 citations)
- JAG: A Crowdsourcing Framework for Joint Assessment and Peer Grading (AAAI 2017 · 6 citations)
What has been tried
100 papers here report 1,012 measured changes (a component added, removed or swapped, compared with a baseline): helped 66%, no clear effect 10%, hurt 20%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add I3 to MiniGPT-4 (FT) | IU X-RAY test · BLEU-1 | 0.389 -> 0.458 | Helped | Bootstrapping Large Language Models for Radiology Report Generation (AAAI 2024 · 72 citations) |
| increase the LLaMA pre-training from none to 7B/13B | HumanML3D validation · FID | 26.01 -> 0.590 -> 0.542 | Helped | MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators (AAAI 2024 · 69 citations) |
| Pretrain on MT-MTR without task descriptions | MoleculeNet classification · average score | 83.86 -> 83.11 | Hurt | MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Prompts (AAAI 2024 · 11 citations) |
| Add chain-of-humor while using the context | TalkFunny · Hum. | 44.8 -> 50.1 | Helped | Talk Funny! A Large-Scale Humor Response Dataset with Chain-of-Humor Interpretation (AAAI 2024 · 8 citations) |
| Add enhanced natural-language event descriptions to ID embeddings | City A and City B · H@1 | 0.2774 -> 0.3176; 0.3056 -> 0.3427 | Helped | Predicting User Behavior in Smart Spaces with LLM-Enhanced Logs and Personalized Prompts (AAAI 2025 · 7 citations) |
| add DICTrank cardiotoxicity keywords to the summary prompt | DICTrank · accuracy | 84.6 -> 88.1; 92.5 -> 90.4; 93.0 -> 93.7 | Mixed | UniTox: Leveraging LLMs to Curate a Unified Dataset of Drug-Induced Toxicity from FDA Labels (NeurIPS 2024 · 7 citations) |
| add self-supervised column-target training | CC18 and CTR23 · test performance | Helped | TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · 6 citations) | |
| Add ANP emotion features | Cyber-MaD — Manifestation · FM | 33.73 -> 35.62 | Helped | Benchmarking Cyber Harassment Dialogue Comprehension through Emotion-Informed Manifestations-Determinants Demarcation (AAAI 2024 · 6 citations) |
| add syntactic similarity retrieval | Rest15 · F1 | 52.58 -> 53.30 | Helped | SimRP: Syntactic and Semantic Similarity Retrieval Prompting Enhances Aspect Sentiment Quad Prediction (AAAI 2025 · 5 citations) |
| Use a long rather than short prompt in zero-shot classification | TREAT-ISE test · weighted F1 | 0.433 -> 0.581 | Helped | Characterizing Information Seeking Events in Health-Related Social Discourse (AAAI 2024 · 5 citations) |
| Retrieve top 10 rather than top 1 document | CFEVER test · recall | 69.05 -> 87.65 | Helped | CFEVER: A Chinese Fact Extraction and VERification Dataset (AAAI 2024 · 4 citations) |
| Use Rationale Extraction instead of Cross-Attn as the Stage I extractor | Combined test set · R@5 | 0.1468 -> 0.3060 | Helped | Deciphering Compatibility Relationships with Textual Descriptions via Extraction and Explanation (AAAI 2024 · 4 citations) |
What reviewers will expect
Datasets and baselines named in the 44 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MNIST · 2 of 44 papers
- CIFAR-10 · 2 of 44 papers
- CIFAR-100 · 2 of 44 papers
- DeepCAD · 2 of 44 papers
- OpenML CC-18 · 2 of 44 papers
Baselines
- GPT-4o · 5 of 44 papers
- LightGBM · 3 of 44 papers
- XGBoost · 3 of 44 papers
- CatBoost · 3 of 44 papers
- Random · 3 of 44 papers
- LLaMA3-8B · 2 of 44 papers
- GPT-4V · 2 of 44 papers
- TabPFN · 2 of 44 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 104 papers; from OpenAlex and the papers' affiliations).
- Stanford University · 9 papers
- Tsinghua University · 5 papers
- Carnegie Mellon University · 4 papers
- University of Science and Technology of China · 3 papers
- Chinese Academy of Sciences · 3 papers
- Nanjing University · 3 papers
- University of Texas, Austin · 3 papers
- Google · 3 papers
- New York University · 3 papers
- University of Washington · 2 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- In-context learning analysis (95 papers)
- Parallel and early-exit decoding (100 papers)
Elsewhere in the topic
- Uncertainty quantification for LLMs (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Safety alignment of LLMs (123 papers)
- Linguistic encoding in LLM representations (123 papers)
- Reasoning mechanisms and failure modes (114 papers)
- Speech and audio language models (111 papers)
- Sign and cross-lingual translation (96 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).