Research map / Large language models / Language, safety and interpretability
LLM unlearning and benchmarks
Evaluate LLMs on unlearning, fallacies, and procedure generalization, with adaptation to domains.
298 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2022–2026) · an approach in Language, safety and interpretability, keyword cluster “prompt · uncertainty · shot”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
84 papers in 2023–24 and 212 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 12.0% to 11.5%.
Open questions the data points to
- Papers report InstructZero on SVAMP (accuracy) anywhere from 0.795 to 0.94: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2022 · Large Language Models are Zero-Shot Reasoners (NeurIPS 2022 · 1,089 citations)
- 2023 · Toolformer: Language Models Can Teach Themselves to Use Tools (NeurIPS 2023 · 507 citations)
- 2024 · Enhancing Job Recommendation through LLM-Based Generative Adversarial Networks (AAAI 2024 · 82 citations)
- 2025 · LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation (AAAI 2025 · 31 citations)
- 2026 · SPRIG: Improving Large Language Model Performance by System Prompt Optimization (ICLR 2026 · 3 citations)
Earliest papers here
- Large Language Models are Zero-Shot Reasoners (NeurIPS 2022 · 1,089 citations)
- P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts (ICLR 2022 · 0 citations)
- Toolformer: Language Models Can Teach Themselves to Use Tools (NeurIPS 2023 · 507 citations)
What has been tried
183 papers here report 2,011 measured changes (a component added, removed or swapped, compared with a baseline): helped 66%, no clear effect 8%, hurt 21%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Replace original self-description with simple resume completion | Designs · map@5 | 0.2627 -> 0.2601 | Hurt | Enhancing Job Recommendation through LLM-Based Generative Adversarial Networks (AAAI 2024 · 82 citations) |
| add collaborative view | Yelp overall · H@10 | 0.6320 -> 0.6673 | Helped | LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation (NeurIPS 2024 · 61 citations) |
| add Zipf, context-sensitivity, and MMD alignment losses | NTU RGB+D 120 X-Set · accuracy | 87.6 -> 91.5 | Helped | LLMs are Good Action Recognizers (CVPR 2024 · 44 citations) |
| Anonymize text using AzureLanguageService | PersonalReddit, location · accuracy | ~86% -> ~55% | Hurt | Beyond Memorization: Violating Privacy via Inference with Large Language Models (ICLR 2024 · 37 citations) |
| Use a different feedback prompt: 'Verify whether your answer is correct, and provide an explanation.' | GSM8K · accuracy | 91.5 -> 91.0; 91.5 -> 91.0 | Mixed | Large Language Models Cannot Self-Correct Reasoning Yet (ICLR 2024 · 36 citations) |
| Keep four initial attention-sink tokens alongside the rolling window instead of evicting all initial tokens | PG19 first book, 65K tokens · perplexity | 5158.07 -> 5.40 | Helped | Efficient Streaming Language Models with Attention Sinks (ICLR 2024 · 36 citations) |
| Add SCFT | Yelp · H@10 | 0.6538 -> 0.6647 | Helped | LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation (AAAI 2025 · 31 citations) |
| Increase the dataset to 40,000 documents | Experiment 1, NameToDescription reverse direction · exact-match accuracy | 0.0 | No clear effect | The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A” (ICLR 2024 · 31 citations) |
| Use SPCC rather than single-description CAD code | Text-to-CAD · ACCcmd | 42.62 -> 80.41 | Helped | CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · 29 citations) |
| Increase PET parameter size by enlarging prompt length or LoRA rank | WMT14 En-De, WMT19 En-Zh, and MLSum · finetuning performance | |α_t| ≪ 1e-2 | No clear effect | When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · 28 citations) |
| Apply reinforcement learning without the synthetic preference dataset | OpenRewriteEval · SARI | 40.92 -> 40.97 | No clear effect | RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting (AAAI 2024 · 25 citations) |
| Use ROUGE-L rather than BLEURT in Algorithm 1 | 14 dataset partitions · accuracy | 78.57% -> 92.86% | Helped | Time Travel in LLMs: Tracing Data Contamination in Large Language Models (ICLR 2024 · 24 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| InstructZero | SVAMP | accuracy | 0.795 – 0.94 | 2 papers, e.g. ICML 2024 and NeurIPS 2025 |
What reviewers will expect
Datasets and baselines named in the 125 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- GSM8K · 11 of 125 papers
- TOFU · 9 of 125 papers
- MMLU · 8 of 125 papers
- PIQA · 6 of 125 papers
- CommonsenseQA · 4 of 125 papers
- MATH · 4 of 125 papers
- C4 · 4 of 125 papers
- TriviaQA · 4 of 125 papers
Baselines
- NPO · 9 of 125 papers
- GPT-4 · 7 of 125 papers
- GPT-4o · 7 of 125 papers
- GA · 7 of 125 papers
- SASRec · 5 of 125 papers
- RMU · 5 of 125 papers
- Claude 3.5 Sonnet · 4 of 125 papers
- KL · 4 of 125 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 296 papers; from OpenAlex and the papers' affiliations).
- Google · 17 papers
- Tsinghua University · 14 papers
- Massachusetts Institute of Technology · 12 papers
- Microsoft · 11 papers
- Stanford University · 10 papers
- Google DeepMind · 9 papers
- National University of Singapore · 8 papers
- Carnegie Mellon University · 8 papers
- Nanyang Technological University · 7 papers
- Alibaba Group · 7 papers
Use it
Download BibTeX of 27 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- In-context learning analysis (95 papers)
- Parallel and early-exit decoding (100 papers)
Elsewhere in the topic
- Uncertainty quantification for LLMs (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Safety alignment of LLMs (123 papers)
- Linguistic encoding in LLM representations (123 papers)
- Reasoning mechanisms and failure modes (114 papers)
- Speech and audio language models (111 papers)
- Sign and cross-lingual translation (96 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).