Research map / Label-efficient and robust learning / Self-supervised and contrastive
A Simple Data Mixing Prior for Improving Self-Supervised Learning
Sucheng Ren, Huiyu Wang, Zhengqi Gao, Shengfeng He, Alan Yuille, Yuyin Zhou, Cihang Xie
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SDMP, treating source-to-mix and related mixed images as weighted positive pairs | MoCo v3 with ViT-S · ImageNet linear evaluation | Top-1 accuracy (%): 73.2% to 73.8% | Improved |
| Add SDMP | DINO with ViT-S · ImageNet linear evaluation | Top-1 accuracy (%): 76.0% to 76.4% | Improved |
| Add SDMP | MoCo v3 with ResNet-50 · ImageNet linear evaluation | Top-1 accuracy (%): 72.8% to 73.5% | Improved |
| Add SDMP | MoCo v3 with ViT-S · ImageNet end-to-end finetuning | Top-1 accuracy: 78.7 to 79.1 | Improved |
| Add SDMP | MoCo v3 with ViT-S · ImageNet semi-supervised, 1% labels | Top-1 accuracy: 54.4 to 55.5 | Improved |
| Add SDMP | DINO with ViT-S · ImageNet-A | Top-1 accuracy (%): 20.0 -> 21.1 | Improved |
| Add SDMP | MoCo v3 with ResNet-50 · CIFAR-10 linear evaluation, 200 epochs | Accuracy: 86.5 -> 89.5 | Improved |
| Add SDMP | MoCo v3 with ResNet-50 · CIFAR-100 linear evaluation, 200 epochs | Accuracy: 63.2 -> 68.2 | Improved |
| Replace sampled source-loss mixing weights with a static weight (ablation) | SDMP with MoCo v3 · ImageNet linear evaluation | Top-1 accuracy: 73.8 to 71.5 | Hurt |
| Replace sampled mixing-loss weights with a static weight (ablation) | SDMP with MoCo v3 · ImageNet linear evaluation | Top-1 accuracy: 73.8 to 72.5 | Hurt |
| Replace independent per-sample mixing coefficients with shared per-batch coefficients (ablation) | SDMP with MoCo v3 · ImageNet linear evaluation | Top-1 accuracy: 73.8 to 72.3 | Hurt |
| Use Mixup alone rather than selecting among Mixup, CutMix, and ResizeMix (ablation) | SDMP with MoCo v3 · ImageNet linear evaluation, 100 epochs | Top-1 accuracy (%): 65.5 to 65.1 | Hurt |
Closest papers by meaning
- Hard Negative Mixing for Contrastive Learning (NeurIPS 2020 · similarity 0.88)
- SMMix: Self-Motivated Image Mixing for Vision Transformers (ICCV 2023 · similarity 0.87)
- Mix-and-Match Tuning for Self-Supervised Semantic Segmentation (AAAI 2018 · similarity 0.86)
- Active Learning by Feature Mixing (CVPR 2022 · similarity 0.86)
- DP-Mix: Mixup-based Data Augmentation for Differentially Private Learning (NeurIPS 2023 · similarity 0.86)
- TiMix: Text-Aware Image Mixing for Effective Vision-Language Pre-training (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
