Research map / Label-efficient and robust learning / Self-supervised and contrastive
Intrinsic Self-Supervision for Data Quality Audits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SelfClean, a data-cleaning method that trains a dataset-specific self-supervised encoder (DINO/SimCLR ViT-tiny) and uses distance-based indicators to rank off-topic samples, near duplicates, and label errors in image datasets.
Problem
Benchmark image datasets contain off-topic images, near duplicates, and label errors that bias performance estimates, and existing cleaning methods are either annotation-dependent or weak on these issue types.
Approach
Combine context-aware self-supervised representation learning on the (noisy) dataset itself with local distance-based indicators: single-linkage agglomerative clustering for off-topic samples, pairwise distances for near duplicates, and an intra/extra-class distance ratio for label errors, plus a statistical procedure for fully automatic decisions.
Key results
- On synthetic 5% contamination across 18 settings, SelfClean with DINO features outperforms competing methods on off-topic, near-duplicate, and label-error detection (e.g., 100.0 AUROC on VDR+BLUR for off-topic).
- On natural label errors, SelfClean achieves 8.4% AP vs 4.3% for the best competitor on ImageNet and 47.8% vs 30.7% on Food-101N.
- Fully automatic mode at alpha=0.10 finds noise below 1% for highly curated datasets and about 14.8% near duplicates in Fitzpatrick17k.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DINO pre-training for SelfClean | SelfClean · Synthetic off-topic, near-duplicate, and label-error benchmarks | AUROC and AP | Helped |
| Use dataset-specific DINO representations instead of ImageNet DINO representations (ablation) | SelfClean · STL-10 synthetic contamination | AP: OT 13.7 -> 27.4; ND 6.1 -> 47.1; LE 69.5 -> 24.8 | Mixed |
| Add multi-crop augmentation (ablation) | SelfClean (DINO) · STL-10, 10% mixed contamination | AP: OT 2.8 -> 27.4; ND 17.5 -> 47.1; LE 39.6 -> 24.8 | Mixed |
| Add color and size augmentations (ablation) | SelfClean (DINO) · STL-10, 10% mixed contamination | AP: OT 4.2 -> 27.4; ND 67.2 -> 47.1; LE 12.6 -> 24.8 | Mixed |
| Clean the evaluation set | DINO-feature kNN classifier · Fitzpatrick17k | macro-F1 change: -4.1^{+1.2}_{-1.3} | Hurt |
| Clean the evaluation set | DINO-feature kNN classifier · DDI | macro-F1 change: +1.2^{+1.9}_{-1.2} | Helped |
| Clean the evaluation set | DINO-feature kNN classifier · ImageNet-1k | macro-F1 change: -0.4^{+0.1}_{-0.2} | Hurt |
| Clean the evaluation set | DINO-feature kNN classifier · Food-101N | macro-F1 change: +0.1^{+0.1}_{-0.1} | Helped |
| Clean the training set | DINO-feature kNN classifier · ImageNet-1k | macro-F1 change: +0.4^{+0.3}_{-0.4} | Helped |
| Clean the training set | DINO-feature kNN classifier · Food-101N | macro-F1 change: +0.1^{+0.2}_{-0.2} | Helped |
Surprising findings
- Dataset-specific DINO pre-training on STL (5,000 samples) outperforms ImageNet-pretrained features for off-topic and near-duplicate detection.
- Cleaning the evaluation set can produce both positive and negative score changes depending on whether wrong samples were misclassified.
Limitations
- Near-duplicate detection does not scale well with dataset size due to pairwise distance computation.
- Requires dataset-specific SSL pre-training, which needs substantial compute.
- Evaluation datasets are already contaminated, so performance measures are capped; synthetic issues may not match real ones.
Experimental setup
Datasets: ImageNet-1k STL-10 CelebA Food-101N CheXpert VinDr-BodyPartXR PatchCamelyon HAM10000 ISIC-2019 Fitzpatrick17k DDI PAD-UFES-20
Benchmarks: STL-10 synthetic contamination VinDr-BodyPartXR synthetic contamination DDI synthetic contamination ImageNet-1k label metadata Food-101N label metadata CelebA identity metadata HAM10000/ISIC-2019 lesion metadata Human expert verification
Metrics: AUROC AP macro-F1
Baselines: HBOS ECOD IForest pHashing SSIM CLearning FastDup NoiseRank
Builds on: DINO SimCLR confident learning
Closest papers by meaning
- Improvements to Context Based Self-Supervised Learning (CVPR 2018 · similarity 0.86)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.83)
- SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification (NeurIPS 2024 · similarity 0.83)
- Measuring Self-Supervised Representation Quality for Downstream Classification Using Discriminative Features (AAAI 2024 · similarity 0.83)
- Systematic Comparison of Semi-supervised and Self-supervised Learning for Medical Image Classification (CVPR 2024 · similarity 0.82)
- Learning to Rank Patches for Unbiased Image Redundancy Reduction (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.