AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Knowledge Distillation as Decontamination? Revisiting the “Data Laundering” Concern

Hengyu Luo, Raúl Vázquez, Timothee Mickus, Filip Ginter, Jörg Tiedemann

ICLR 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Distillation with causal and adaptive guidance

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
distill through a student instead of training directly on contaminated dataDistilBERT · tweet sentimentaccuracy gain from test contamination: 25.66% -> 3.25%Improved
distill through a student instead of training directly on contaminated dataDistilBERT · 20newsgroupsaccuracy gain from test contamination: 11.91% -> 1.42%Improved
contaminated teacher distilled into clean student (laundering)DistilBERT student · AGNewsaccuracy gain: 0.65% gain, significantMixed
contaminated teacher distilled into clean student (laundering)DistilBERT student · banking77, rotten tomatoes, SNLIaccuracy gainNo clear effect
replace BERT teacher with Llama3.2-1B (ablation)clean student · AGNewsaccuracy: 91.59 -> 91.08 (-0.51)No clear effect
replace BERT teacher with Qwen3-0.6B (ablation)clean student · rotten tomatoesaccuracy: 83.77 -> 84.22 (+0.45)No clear effect
use stronger LLM teacher with contaminated data (ablation)DistilBERT student · tweet sentimentstudent accuracy gain: teacher gains 25.80%, student gain limited to 1.91%No clear effect
use mixed objectives or soft labels instead of pure hard distillation (ablation)DistilBERT student · eight benchmarkslaundering gainMixed
compare laundering with contamination at sample levelsample-level effects · agnews and tweet sentimentPearson correlation: 0.32 and 0.31No clear effect
compare laundering with contamination at sample levelsample-level effects · SNLIPearson correlation: -0.03No clear effect
widen train-test distributional gap (Level 1 to 5) (ablation)student laundering gain · rotten tomatoesaccuracy gain: 0.4 -> 6.5Improved
widen train-test distributional gap (Level 1 to 5) (ablation)student laundering gain · emotionstatistical significance: p-values 0.1455 to 0.0001 range across levelsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy