Research map / Large language models / Instruction and fine-tuning
Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
Yongding Tao, Tian Wang, Yihong Dong, Huanyu Liu, Kechi Zhang, Hu XiaoLong, Ge Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use self-critique probing with entropy similarity instead of best baseline | contamination detectors · RL-MIA (Qwen2.5-7B-Instruct) | average AUC: 0.62 -> 0.70 | Improved |
| use self-critique probing with entropy similarity instead of best baseline | contamination detectors · RL-MIA (DeepSeek-Math-7B-Instruct) | average AUC: 0.55 -> 0.64 | Improved |
| likelihood-based detectors (PPL, Min-K%, Min-K%++) for RL-phase contamination | detectors · RL-MIA | AUC: 0.44 to 0.53 average AUC | No clear effect |
| replace CDD sampling probe with entropy (Entropy-Temp) (ablation) | CDD · RL-MIA (Qwen2.5-7B-Instruct) | average AUC: 0.51 -> 0.56 | Improved |
| replace Recall metric with entropy (Entropy-Noise) (ablation) | Recall · RL-MIA (Qwen2.5-7B-Instruct) | average AUC: 0.59 -> 0.62 | Improved |
| replace Recall metric with entropy (Entropy-Noise) (ablation) | Recall · RL-MIA (DeepSeek-Math-7B-Instruct) | average AUC: 0.54 -> 0.55 | Improved |
| use self-critique probing instead of non-member prefix or temperature sampling for entropy detectors (ablation) | Entropy-Temp/Entropy-Noise · RL-MIA | average AUC: +13% (Qwen2.5-7B-Instruct), +16% (DeepSeek-Math-7B-Instruct) | Improved |
| train with different RL algorithms (PPO, GRPO, DAPO) (ablation) | Self-Critique · K&K (Qwen2.5-3B-Instruct) | AUC: 0.61, 0.61, 0.60 | No clear effect |
| reduce Top-K entropy approximation from K=50 to K=3 (ablation) | Self-Critique · AIME25 (Qwen2.5-7B-Instruct) | AUC: 0.7156 -> 0.7022 | No clear effect |
| reduce Top-K entropy approximation from K=50 to K=3 (ablation) | Self-Critique · K&K (Qwen2.5-7B-Instruct) | AUC: 0.6584 -> 0.6460 | No clear effect |
| restrict to lower-pretraining-contamination subset (ablation) | Self-Critique · GSM8K (Qwen2.5-0.5B-Instruct, PPO) | AUC | Improved |
| random-control subset of same size (ablation) | Self-Critique · GSM8K (Qwen2.5-0.5B-Instruct, PPO) | AUC | No clear effect |
Closest papers by meaning
- Time Travel in LLMs: Tracing Data Contamination in Large Language Models (ICLR 2024 · similarity 0.90)
- Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination (AAAI 2026 · similarity 0.89)
- Behavior Injection: Preparing Language Models for Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.88)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.88)
- Beyond Verifiable Rewards: Scaling Reinforcement Learning in Language Models to Unverifiable Data (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
