Research map / Large language models / Language, safety and interpretability
Time Travel in LLMs: Tracing Data Contamination in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes guided-instruction completion of dataset instances plus two partition-level rules (bootstrap BLEURT/ROUGE-L test and GPT-4 few-shot ICL match counting) to detect LLM training-data contamination without access to pre-training data.
Problem
Detecting whether downstream test/train data from benchmark datasets has been ingested into closed LLMs' pre-training data, when pre-training data is inaccessible and compute is limited.
Approach
Prompt the LLM with a guided instruction (dataset name, split name, random-length prefix, label) to complete a reference instance, then compare overlap with a general instruction lacking dataset/split identifiers; flag partitions via a bootstrap significance test or a GPT-4 few-shot ICL classifier counting exact/near-exact matches.
Key results
- GPT-4 ICL-based Algorithm 2 matches human judgment in 14/14 GPT-4 partitions (100%) and 13/14 GPT-3.5 partitions (92.86%).
- Algorithm 1 with ROUGE-L reaches 13/14 on GPT-4 but only 7/14 (50%) on GPT-3.5; BLEURT reaches 11/14 and 9/14.
- Human and GPT-4 ICL evaluation find GPT-4 contaminated with AG News, WNLI, and XSum test partitions, and with IMDB and RTE train partitions; intentional contamination of GPT-3.5 yields exact matches on all three tested datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPT-4 few-shot ICL to judge guided completions instead of overlap-score heuristics | Contamination detection with GPT-4 as the investigated model · 14 dataset partitions | accuracy: 78.57% -> 100.00% | Helped |
| Use GPT-4 few-shot ICL to judge guided completions instead of BLEURT overlap scores | Contamination detection with GPT-3.5 as the investigated model · 14 dataset partitions | accuracy: 64.29% -> 92.86% | Helped |
| Use ROUGE-L rather than BLEURT in Algorithm 1 (ablation) | Contamination detection with GPT-4 as the investigated model · 14 dataset partitions | accuracy: 78.57% -> 92.86% | Helped |
| Use ROUGE-L rather than BLEURT in Algorithm 1 (ablation) | Contamination detection with GPT-3.5 as the investigated model · 14 dataset partitions | accuracy: 64.29% -> 50.00% | Hurt |
| Use guided instruction with dataset and partition names instead of general instruction (ablation) | GPT-4 completion overlap scoring · AG News train, BLEURT | BLEURT: 0.63 -> *0.70 | Helped |
| Use guided instruction with dataset and partition names instead of general instruction (ablation) | GPT-4 completion overlap scoring · IMDB train, BLEURT | BLEURT: 0.43 -> 0.48 | Hurt |
| Use guided instruction with dataset and partition names instead of general instruction (ablation) | GPT-3.5 completion overlap scoring · AG News train, ROUGE-L | ROUGE-L: 0.06 -> *0.16 | Helped |
| Use guided instruction with dataset and partition names instead of general instruction (ablation) | GPT-3.5 completion overlap scoring · RTE train, ROUGE-L | ROUGE-L: 0.37 -> 0.32 | Hurt |
Surprising findings
- ChatGPT-Cheat? labels every GPT-4 partition as suspicious, so its strict evaluation scores 0/14 on GPT-4.
- Algorithm 1 with ROUGE-L performs at chance level (7/14) on GPT-3.5 despite strong GPT-4 results.
Limitations
- Does not distinguish between types or sources of contamination (direct instances vs metadata).
- Relies on only 10 sampled instances per partition, and near-exact match thresholds are heuristic.
- Evaluation covers only seven datasets and two OpenAI models; ground truth is based on human judgment.
Experimental setup
Datasets: IMDB AG News Yelp Full Reviews SAMSum XSum WNLI RTE GSM8k
Benchmarks: IMDB AG News Yelp Full Reviews RTE WNLI SAMSum XSum
Metrics: success rate vs human judgment accuracy ROUGE-L BLEURT
Baselines: ChatGPT-Cheat? Algorithm 1 (BLEURT) Algorithm 1 (ROUGE-L)
Builds on: ChatGPT-Cheat? few-shot in-context learning BLEURT ROUGE-L
Closest papers by meaning
- Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination’s Impact on Machine Translation (ICML 2025 · similarity 0.92)
- Task Contamination: Language Models May Not Be Few-Shot Anymore (AAAI 2024 · similarity 0.91)
- The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (ICML 2025 · similarity 0.90)
- How Much Can We Forget about Data Contamination? (ICML 2025 · similarity 0.90)
- ConStat: Performance-Based Contamination Detection in Large Language Models (NeurIPS 2024 · similarity 0.90)
- Training on the Benchmark Is Not All You Need (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.