Research map / Large language models / Instruction and fine-tuning
How Contaminated Is Your Benchmark? Measuring Dataset Leakage in Large Language Models with Kernel Divergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Kernel Divergence Score (KDS), which measures benchmark contamination by how much LoRA fine-tuning changes the RBF kernel similarity matrix of sample embeddings, and shows near-perfect monotonicity on controlled seen/unseen mixtures.
Problem
Measuring how much an evaluation benchmark has overlapped with an LLM's pre-training data at the dataset level, when the pre-training corpus is unavailable, with scores that are monotonic in contamination rate and consistent across subsets.
Approach
Fine-tuning (LoRA SFT) on a benchmark changes embedding relationships among unseen samples more than among seen ones; KDS computes a normalized, soft-gated divergence between the RBF kernel matrices of embeddings before and after fine-tuning, and negates it so higher means more contamination.
Key results
- KDS achieves average Spearman 0.990 and Pearson 0.982 for monotonicity across WikiMIA, BookMIA, and ArxivTection on Mistral-7B-Instruct-v0.2, the highest average among baselines (best baseline Perplexity 0.966 Spearman).
- On six Pile subsets KDS averages Spearman 0.944 and Pearson 0.948; Zlib+FSD and Perplexity+FSD reach 0.868 and 0.836 on average.
- Removing the fine-tuning component drops WikiMIA Spearman from 0.999 to 0.683, while removing soft gating barely changes it (0.998).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Kernel Divergence Score instead of the strongest baseline | Mistral-7B-Instruct-v0.2 contamination scoring · WikiMIA | Spearman correlation: 0.979 -> 0.999 | Helped |
| Use Kernel Divergence Score instead of Min-K% | Mistral-7B-Instruct-v0.2 contamination scoring · BookMIA | Spearman correlation: 0.998 -> 0.997 | Hurt |
| Use Kernel Divergence Score instead of Perplexity | Mistral-7B-Instruct-v0.2 contamination scoring · ArxivTection | Spearman correlation: 1.000 -> 0.975 | Hurt |
| Use Kernel Divergence Score instead of the strongest baseline | Mistral-7B-Instruct-v0.2 contamination scoring · Average across WikiMIA, BookMIA, and ArxivTection | Spearman correlation: 0.966 -> 0.990 | Helped |
| Remove soft gating (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.999 -> 0.998 | Hurt |
| Add fine-tuning-induced pairwise distance changes (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.683 -> 0.999 | Helped |
| Replace RBF with Euclidean Distance Kernel (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.999 -> 0.999 | No clear effect |
| Replace RBF with Cosine Similarity Kernel (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.999 -> 0.998 | No clear effect |
| Replace RBF with Dot-product Kernel (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.999 -> 0.998 | No clear effect |
| Set kernel bandwidth γ to 0.001, 0.01, 0.1, or 1.0 instead of the median heuristic (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.999 -> 0.999 | No clear effect |
| Set kernel bandwidth γ to 10.0 instead of the median heuristic (ablation) | Kernel Divergence Score · WikiMIA | Spearman correlation: 0.999 -> 0.801 | Hurt |
| Replace stochastic GD with batch GD (ablation) | Kernel Divergence Score with 1 fine-tuning epoch · WikiMIA | Spearman correlation: 0.999 -> 0.908 | Hurt |
Surprising findings
- Many existing per-sample baselines (e.g., Zlib, Perplexity, Min-K%) yield negative correlations on BookMIA and Min-K%++ on WikiMIA, indicating reversed signs.
- Adding FSD to several baselines reverses or degrades correlation on BookMIA, despite FSD being designed to use fine-tuning information.
Limitations
- Kernel computation is quadratic in dataset size, which can become a time and memory bottleneck for large datasets.
- Evaluation relies on controlled seen/unseen mixtures from MIA benchmarks that the paper notes may have temporal shift issues; the Pile-based check is the only alternative.
- Contamination is evaluated only in a controlled setting with known seen/unseen labels; no real-world benchmark contamination ground truth is validated.
Experimental setup
Datasets: WikiMIA BookMIA ArxivTection The Pile subsets (Wikipedia, PhilPapers, Enron, HackerNews, Pile-CC, StackExchange)
Benchmarks: WikiMIA BookMIA ArxivTection The Pile subsets
Metrics: Spearman correlation Pearson correlation MAPE (consistency)
Baselines: Zlib Zlib + FSD Perplexity Perplexity + FSD Min-K% Min-K% + FSD Min-K%++ Min-K%++ + FSD SRCT
Builds on: Fine-tuned Score Deviation Sharded Rank Comparison Test Min-K% Prob LoRA
Closest papers by meaning
- Time Travel in LLMs: Tracing Data Contamination in Large Language Models (ICLR 2024 · similarity 0.89)
- Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination’s Impact on Machine Translation (ICML 2025 · similarity 0.88)
- ConStat: Performance-Based Contamination Detection in Large Language Models (NeurIPS 2024 · similarity 0.88)
- The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (ICML 2025 · similarity 0.88)
- How Much Can We Forget about Data Contamination? (ICML 2025 · similarity 0.87)
- Training on the Benchmark Is Not All You Need (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.