Research map / Large language models / Language, safety and interpretability
Rethinking Pretraining Data Detection for LLMs: From Local to Global
Chenye Ke, Yan Zhuang, Zirui Liu, Qi Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-information calibration and volatility divergence to NLL volatility (ablation) | AECA · WikiMIA+MIMIR (Pythia-2.8B/6.9B) | average AUC: 0.606 -> 0.615 | Improved |
| add entropic convolution on top of calibration (ablation) | AECA · WikiMIA+MIMIR (Pythia-2.8B/6.9B) | average AUC: 0.615 -> 0.619 | Improved |
| raise volatility coefficient lambda from 0 to 5 (ablation) | AECA · WikiMIA length 128, Pythia-12B | AUC: 0.582 -> 0.716 | Improved |
| raise volatility coefficient lambda from 0 to 5 (ablation) | AECA · WikiMIA length 128, GPT-Neo-2.7B | AUC: 0.599 -> 0.698 | Improved |
| raise lambda beyond 5 up to 10 (ablation) | AECA · WikiMIA length 128, Pythia-12B | AUC: 0.716 -> 0.714 | No clear effect |
| vary Laplace smoothing alpha from 0.01 to 3 (ablation) | AECA · WikiMIA length 128, Pythia-12B | AUC: 0.716 and 0.717 | No clear effect |
| use small cross-domain Case-law reference corpus instead of large C4 (ablation) | AECA · WikiMIA, Pythia-12B | AUC: 0.716 -> 0.705 | Hurt |
| use larger 10GB C4 reference corpus instead of 1GB (ablation) | AECA · WikiMIA, Pythia-12B | AUC: 0.711 -> 0.716 | Improved |
| replace PPL with AECA | GPT-NeoX-20B · WikiMIA length 256 (original) | AUC: 0.713 -> 0.758 | Improved |
| replace Min-K% with AECA | GPT-NeoX-20B · WikiMIA length 256 (original) | AUC: 0.743 -> 0.758 | Improved |
| replace best baseline with AECA | MIMIR average · MIMIR three subsets | average AUC: 0.540 -> 0.546 | Improved |
| replace Min-K%++ with AECA at length 256 | WikiMIA length 256 · WikiMIA length 256 (average) | average AUC: 0.619 -> 0.714 | Improved |
Closest papers by meaning
- Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.92)
- Detecting Pretraining Data from Large Language Models (ICLR 2024 · similarity 0.91)
- Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection (ICLR 2025 · similarity 0.88)
- DALD: Improving Logits-based Detector without Logits from Black-box LLMs (NeurIPS 2024 · similarity 0.88)
- To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models (ICML 2024 · similarity 0.87)
- Copyright Traps for Large Language Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
