Research map / Label-efficient and robust learning / Knowledge distillation
SuperMix: Supervising the Mixing Data Augmentation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RandAugment after SuperMix | SuperMix classification training · CIFAR-100 and ImageNet | Top-1 accuracy: 79.7 -> 79.9; 83.6 -> 83.9; 85.5 -> 85.8; 77.6 -> 78.2; 80.8 -> 81.3 | Helped |
| Use teacher-supervised mixing instead of blind mixing | Knowledge distillation · CIFAR-100 | Student Top-1 accuracy: 76.93 vs 76.40; 76.11 vs 75.85; 72.64 vs 72.33; 72.75 vs 72.68; 74.80 vs 74.24; 77.16 vs 76.81; 75.38 vs 74.87 | Helped |
| Add SuperMix images to the KD objective | Knowledge distillation · CIFAR-100 | Student Top-1 accuracy: 77.45 vs 76.52; 76.53 vs 75.70; 73.19 vs 72.22; 72.96 vs 72.23; 75.21 vs 74.24; 77.59 vs 76.46; 76.03 vs 75.02 | Helped |
| Increase mixed dataset size (ablation) | SuperMix distillation · CIFAR-100 | Top-1 test accuracy | Helped |
| Use SuperMix with k=3 instead of k=2 (ablation) | SuperMix distillation · CIFAR-100 | Distillation performance | Hurt |
| Use SuperMix with k=3 instead of k=2 (ablation) | SuperMix distillation · ImageNet | Student Top-1 accuracy: 73.42 -> 73.65 | Helped |
| Increase the mixing-coefficient parameter α (ablation) | SuperMix distillation · CIFAR-100 | Distillation performance | Helped |
| Increase sparsity-loss weight up to approximately 30 (ablation) | SuperMix mask optimization · CIFAR-100 | Distillation performance | Helped |
| Increase sparsity-loss weight beyond approximately 30 (ablation) | SuperMix mask optimization · CIFAR-100 | Mask accuracy | Hurt |
| Use smooth projection in Newton mask optimization (ablation) | SuperMix · ImageNet mask optimization | Execution time: 15.41 -> 0.23; 4.25 -> 0.06 | Helped |
| Use smooth projection rather than Newton optimization without SP (ablation) | SuperMix · ImageNet mask optimization | Execution time: 6.59 -> 0.23; 1.98 -> 0.06 | Helped |
| Increase the number of input images for MixUp or CutMix beyond 2 (ablation) | Blind mixing augmentation for distillation · CIFAR-100 and ImageNet | Distillation performance | Hurt |
Closest papers by meaning
- Adversarial AutoMixup (ICLR 2024 · similarity 0.88)
- TransMix: Attend To Mix for Vision Transformers (CVPR 2022 · similarity 0.86)
- DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models (CVPR 2024 · similarity 0.85)
- Supervision Interpolation via LossMix: Generalizing Mixup for Object Detection and Beyond (AAAI 2024 · similarity 0.84)
- AGMixup: Adaptive Graph Mixup for Semi-supervised Node Classification (AAAI 2025 · similarity 0.83)
- TdAttenMix: Top-Down Attention Guided Mixup (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.