Research map / Large language models / Language, safety and interpretability
Statistical frameworks for LLM evaluation
Use conformal prediction, ranking models, and noise analysis for valid LLM evaluation.
124 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Language, safety and interpretability, keyword cluster “safety · jailbreak · harmful”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
23 papers in 2023–24 and 96 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.3% to 5.2%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · The Curious Case of Neural Text Degeneration (ICLR 2020 · 1,122 citations)
- 2021 · Is Automated Topic Model Evaluation Broken? The Incoherence of Coherence (NeurIPS 2021 · 23 citations)
- 2022 · Factuality Enhanced Language Models for Open-Ended Text Generation (NeurIPS 2022 · 55 citations)
- 2023 · Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias (NeurIPS 2023 · 72 citations)
- 2024 · Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · 94 citations)
- 2025 · SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses (AAAI 2025 · 4 citations)
- 2026 · Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity (ICML 2026 · 2 citations)
Earliest papers here
- The Curious Case of Neural Text Degeneration (ICLR 2020 · 1,122 citations)
- Is Automated Topic Model Evaluation Broken? The Incoherence of Coherence (NeurIPS 2021 · 23 citations)
- Perception Score: A Learned Metric for Open-ended Text Generation Evaluation (AAAI 2021 · 13 citations)
What has been tried
55 papers here report 574 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 7%, hurt 22%, mixed 8%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use different persona-based initialization prompts for agents | MMLU · accuracy | 71.1 to 74.2 | Helped | Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · 94 citations) |
| Add a one-shot example | 5-category content moderation benchmark · Micro-F1 | 0.65 -> 0.70 | Helped | LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset (ICLR 2024 · 26 citations) |
| Remove option IDs (ask model to pick option content via likelihood) | MMLU/ARC/CSQA 0-shot · RStd (selection bias) | -6.4 MMLU, -5.1 ARC, -6.2 CSQA (avg RStd change) | Helped | Large Language Models Are Not Robust Multiple Choice Selectors (ICLR 2024 · 23 citations) |
| Add the score rubric | Vicuna Bench · Pearson correlation | 0.355 -> 0.457 | Helped | Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models (ICLR 2024 · 14 citations) |
| Add duplicate rejection to First-K scoring | Long-output tasks · output set size | No clear effect | Conformal Language Modeling (ICLR 2024 · 7 citations) | |
| Add 3-step CoT | MLLM-as-a-Judge selected subset · Pearson similarity | 0.557 -> 0.299 | Hurt | MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark (ICML 2024 · 5 citations) |
| Double the discrimination-phase in-context examples | GSM8K, TriviaQA, MT-Bench · DG-Diff | -2.8 -> -1.4; -5.5 -> -5.9; -5.9 -> -5.8; -0.1 -> -0.4; 0.0 -> 0.1; -1.6 -> -1.7; -0.13 -> -0.09; -0.51 -> -0.53; -0.17 | Mixed | SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses (AAAI 2025 · 4 citations) |
| Add BOS as the context separator rather than using no separator | Wikipedia, entropy band [2.0, 3.0) · AUROC | 0.56 -> 0.68 | Helped | Distinguishing the Knowable from the Unknowable with Language Models (ICML 2024 · 3 citations) |
| Increase fairness penalty lambda | MovieLens-1M, iteration 3 · #Violations | 18 (lambda=0.1) -> 2 (lambda=0.7); 2 (lambda=0.7) at 0.9 | Helped | FACTER: Fairness-Aware Conformal Thresholding and Prompt Engineering for Enabling Fair LLM-Based Recommender Systems (ICML 2025 · 2 citations) |
| use distance-aware GeoMLE instead of vanilla MLE | QA tasks (Llama-2-7B and Llama-2-13B) · AUROC | 0.758 -> 0.767 (CoQA, Llama-2-7B); 0.748 -> 0.775 (TriviaQA 0-shot, Llama-2-13B) | Helped | Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension (ICML 2024 · 2 citations) |
| Use the MixEval sample-level mixture instead of benchmark-level or uniform mixture | Arena Elo · correlation | Helped | MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures (NeurIPS 2024 · 2 citations) | |
| Increase training examples per value from 10 or 20 to 50 or 100 | Social Risks and Schwartz Value; original and generalized splits · accuracy | Mixed | CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses (NeurIPS 2024 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 56 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MMLU · 8 of 56 papers
- TruthfulQA · 7 of 56 papers
- GSM8K · 6 of 56 papers
- TriviaQA · 5 of 56 papers
- SciQ · 4 of 56 papers
- MT-Bench · 3 of 56 papers
- GPQA · 3 of 56 papers
- Natural Questions · 3 of 56 papers
Baselines
- GPT-4o · 4 of 56 papers
- DeepSeek R1 · 3 of 56 papers
- Semantic Entropy · 3 of 56 papers
- P(True) · 3 of 56 papers
- Perplexity · 2 of 56 papers
- GPT-3.5-turbo · 2 of 56 papers
- GPT-4-turbo · 2 of 56 papers
- top-p sampling · 2 of 56 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 119 papers; from OpenAlex and the papers' affiliations).
- Stanford University · 7 papers
- Peking University · 5 papers
- Carnegie Mellon University · 5 papers
- Massachusetts Institute of Technology · 4 papers
- University of Washington · 4 papers
- University of Southern California · 4 papers
- Google · 4 papers
- Amazon · 4 papers
- Meta · 3 papers
- Shanghai Jiao Tong University · 3 papers
Use it
Download BibTeX of 29 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Uncertainty quantification for LLMs (136 papers)
- Bias measurement and mitigation (59 papers)
- Jailbreak elicitation and risk (21 papers)
Elsewhere in the topic
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Safety alignment of LLMs (123 papers)
- Linguistic encoding in LLM representations (123 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).