Research map / Large language models / Language, safety and interpretability
Counterfactual Memorization in Neural Language Models
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, Nicholas Carlini
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| count more near-duplicates of an example | counterfactual memorization score · RealNews | memorization score: Pearson -0.39 | Hurt |
| train for more epochs | LM memorization · RealNews | memorization: 59% of examples consistently increasing | Improved |
| deduplicate C4 with NearDup (removes ~3%) (ablation) | C4 memorization distribution · C4 | memorization distribution | No clear effect |
| increase number of models used for estimation from 96 to 192 (ablation) | memorization ranking · all datasets | Spearman R vs 400 models: at least 0.992 at 192 models | Improved |
Closest papers by meaning
- Mitigating Memorization in Language Models (ICLR 2025 · similarity 0.89)
- Memorization Sinks: Isolating Memorization during LLM Training (ICML 2025 · similarity 0.87)
- Rethinking LLM Memorization through the Lens of Adversarial Compression (NeurIPS 2024 · similarity 0.87)
- Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon (ICLR 2025 · similarity 0.87)
- Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data (ICLR 2025 · similarity 0.87)
- Counterfactual reasoning: an analysis of in-context emergence (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
