Research map / Large language models / Instruction and fine-tuning
Fine-tuning can Help Detect Pretraining Data from Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fine-tuned Score Deviation (FSD), which fine-tunes an LLM on a few unseen same-domain non-member texts and uses the drop in existing membership scores (Perplexity, Min-k%, Zlib, Lowercase) as a pretraining-data detector.
Problem
Detecting whether a text was in an LLM's pretraining data is hard because likelihood-based scoring functions overlap heavily between members and non-members, giving suboptimal AUC and TPR@5%FPR.
Approach
Fine-tune the target LLM with LoRA on a few post-release (unseen) same-domain non-member texts; non-member scores drop much more than member scores, so the difference between pre-fine-tuning and post-fine-tuning scores (FSD) enlarges the member/non-member gap. Works as a wrapper around existing scoring functions.
Key results
- On WikiMIA with OPT-6.7B, FSD raises Min-k% AUC from 0.62 to 0.91.
- On ArXivTection with LLaMA-7B, FSD raises TPR@5%FPR for Perplexity from 0.10 to 0.81.
- On BookMIA with LLaMA-7B, FSD with Zlib reaches 98.6% accuracy, versus a 71.8% improvement over the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply FSD to likelihood-based scoring functions | LLaMA-7B detection methods · WikiMIA | AUC: 0.64 -> 0.92 | Helped |
| Apply FSD to scoring functions | LLaMA-7B detection methods · ArXivTection | AUC: 0.68 -> 0.92 | Helped |
| Use FSD for Zlib book detection | LLaMA-7B · BookMIA | accuracy: 57.4 -> 98.6 | Helped |
| Use non-members rather than members for fine-tuning (ablation) | FSD with LLaMA-7B · WikiMIA | AUC: 0.78 -> 0.92 | Helped |
| Fine-tune on a mixed member and non-member dataset instead of non-members (ablation) | FSD with LLaMA-7B and Perplexity · WikiMIA | AUC: 0.92 -> 0.68 | Hurt |
| Use AdaLoRA instead of LoRA for fine-tuning (ablation) | FSD with LLaMA-7B and Perplexity · WikiMIA | AUC: 0.92 -> 0.82 | Hurt |
| Use IA3 instead of LoRA for fine-tuning (ablation) | FSD with LLaMA-7B and Perplexity · WikiMIA | AUC: 0.92 -> 0.91 | No clear effect |
| Fine-tune using non-members from an unrelated domain (ablation) | FSD with LLaMA-7B and Perplexity · ArXivTection | AUC: 0.68 -> 0.52 | Hurt |
| Fine-tune using non-members from a mix of domains (ablation) | FSD with LLaMA-7B and Perplexity · ArXivTection | AUC: 0.68 -> 0.93 | Helped |
| Apply FSD after removing timestamps (ablation) | LLaMA-7B Perplexity detection · WikiMIA Deletion | AUC: 0.62 -> 0.76 | Helped |
| Apply FSD after replacing timestamp years with 2023 (ablation) | LLaMA-7B Perplexity detection · WikiMIA Replacement | AUC: 0.54 -> 0.71 | Helped |
| Use FSD with 100 non-member fine-tuning examples (ablation) | Perplexity-based detection · WikiMIA | AUC: 0.63 -> 0.91 | Helped |
Surprising findings
- Fine-tuning on members instead of non-members yields a weaker improvement, and fine-tuning on non-members from an unrelated domain (WikiMIA fine-tuning evaluated on ArXivTection) fails to help.
- Replacing years in timestamps (Replacement) drops baselines to near random (e.g., Perplexity 0.54 AUC), though FSD still reaches 0.71.
Limitations
- Requires collecting a few same-domain unseen non-member texts, which may be hard for diverse test sets.
- Effectiveness depends on non-member data quality and domain match; cross-domain fine-tuning fails on ArXivTection.
- Relies on white-box access to fine-tune the model and output probabilities.
Experimental setup
Datasets: WikiMIA ArXivTection BookMIA BookTection Pile
Benchmarks: WikiMIA ArXivTection BookMIA BookTection Pile
Metrics: AUC TPR@5%FPR accuracy
Baselines: Perplexity Zlib Lowercase Min-k%
Builds on: Min-k% LoRA Perplexity-based membership inference Zlib ratio Lowercase ratio
Closest papers by meaning
- Improving Pretraining Data Using Perplexity Correlations (ICLR 2025 · similarity 0.91)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.90)
- An Empirical Study on Noisy Data and LLM Pretraining Loss Divergence (NeurIPS 2026 · similarity 0.89)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.89)
- Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.89)
- Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.