Research map / Large language models / Language, safety and interpretability

ConStat: Performance-Based Contamination Detection in Large Language Models

NeurIPS 2024 · Large language models › Language, safety and interpretability · approach: Data and hallucination detection

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ConStat, a statistical test that defines contamination as performance inflated relative to reference models on a primary benchmark versus similar reference benchmarks (rephrased, synthetic, or different benchmark), and detects contamination in popular LLMs.

Problem

Existing contamination detection methods rely on detecting benchmark samples in training data, are easy to evade, and cannot quantify the effect of contamination on measured performance.

Approach

Define contamination as artificially inflated, non-generalizing benchmark performance relative to reference models; map a model's reference-benchmark score to the expected original-benchmark score via a smoothed hardness correction function fitted on reference models, and test the residual with bootstrapped pivotal confidence bounds.

Key results

• ConStat detects 89.1% of syntax-specific and 98.4% of sample-specific contaminated finetuned models, versus at most 84.8% and 70.5% for baselines even under oracle thresholds.
• Estimated sample-specific contamination matches ground truth with r^2 = 0.94 on the controlled finetuned models.
• Top-3 Open LLM Leaderboard 7B models (mistroll, yampeleg, multiverse) show benchmark-specific contamination with estimated effects above 10% on GSM8k, HellaSwag, and ARC, and Yi-34B shows significant sample-specific contamination on ARC and HellaSwag.

Measured effects reported in the paper

Surprising findings

• Several widely used reference models, including Llama-3-70B and Llama-2-Instruct-70B, show significant sample-specific contamination, and Mistral-7B-v0.1 was removed from the reference set for this reason.
• Phi-2 and Phi-3 mini show very large benchmark-specific contamination effects (>15%) on GSM8k, which the authors attribute to reasoning-focused training and small size with multiple-choice MathQA.

Limitations

• If reference models are themselves contaminated, the measured effect is relative to a contaminated baseline.
• Synthetic reference samples generated by an LLM may be easier for that generating model, making its own results unreliable for sample-specific detection.
• Benchmark-specific contamination is hard to interpret because small differences between benchmarks affect performance.

Experimental setup

Datasets: GSM8k ARC-Challenge MMLU HellaSwag MathQA SciQ LAMBADA-OpenAI OpenOrca

Benchmarks: GSM8k ARC-Challenge MMLU HellaSwag

Metrics: estimated contamination effect (delta-hat) p-value true positive rate coefficient of determination r^2 lower confidence bound

Baselines: perplexity-based contamination detection Min-K% prob Chen et al. benchmark-matching detection rephrased-benchmark evaluation (Yang et al.) Mean-Test Normalized-Test

Builds on: Benjamini-Hochberg procedure LM Evaluation Harness

Closest papers by meaning

• Time Travel in LLMs: Tracing Data Contamination in Large Language Models (ICLR 2024 · similarity 0.90)
• Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination’s Impact on Machine Translation (ICML 2025 · similarity 0.90)
• The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination (ICML 2025 · similarity 0.90)
• Provable Joint Decontamination for Benchmarking Multiple Large Language Models (NeurIPS 2026 · similarity 0.88)
• Task Contamination: Language Models May Not Be Few-Shot Anymore (AAAI 2024 · similarity 0.88)
• How Contaminated Is Your Benchmark? Measuring Dataset Leakage in Large Language Models with Kernel Divergence (ICML 2025 · similarity 0.88)