Research map / Label-efficient and robust learning / Recognition and long-tail
Revisiting the Critical Factors of Augmentation-Invariant Representation Learning
Junqiang Huang, Xiangwen Kong, Xiangyu Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add symmetric contrastive loss (ablation) | MoCo v2 (with BN, predictor, momentum ascent) · ImageNet linear eval | top-1 accuracy (%): 2.0% accuracy increment | Improved |
| add all BYOL-style model configurations and complex augmentations (ablation) | MoCo v2 · ImageNet linear eval (200 ep) | top-1 accuracy (%): 72.0 -> 72.4 | Improved |
| add the BYOL-style configurations (MoCo v2+) (ablation) | MoCo v2 · transfer (VOC, COCO) | AP: about 0.5% improvement | No clear effect |
| pretrain with LARS instead of SGD (ablation) | MoCo v2+ · VOC07 AP50 | AP50 (%): 77.1 -> 62.9 | Hurt |
| pretrain with LARS instead of SGD (ablation) | BYOL · VOC07 AP50 | AP50 (%): 76.2 -> 71.7 | Hurt |
| add NormRescale to LARS-trained BYOL | BYOL-LARS · VOC07 AP50 | AP50 (%): 71.7 -> 76.6 | Improved |
| use a constant 0.1 norm scale instead of SGD anchor in NormRescale (ablation) | BYOL-NR · VOC07 AP50 | AP50 (%): 76.6 -> 76.1 | No clear effect |
| symmetrize the network (add teacher predictor) (ablation) | MoCo v2+ · ImageNet linear eval (200 ep) | top-1 accuracy (%): 72.4 -> 71.2 | Hurt |
| symmetrize the network (add teacher predictor) (ablation) | MoCo v2+ · VOC07, COCO, CityScapes transfer | AP/mIoU: gap within 0.4 | No clear effect |
| symmetrize the network (add teacher predictor) (ablation) | MoCo v2+ · CIFAR-10-LT and CIFAR-100-LT | linear and fine-tuning accuracy | Improved |
| replace negative-free BYOL with contrastive loss (ablation) | BYOL · CIFAR-10-LT and CIFAR-100-LT | accuracy | Improved |
| remove data augmentations with color transforms (ablation) | MoCo v2+, S-MoCo v2+, BYOL · ImageNet linear eval | top-1 accuracy (%) | Hurt |
Closest papers by meaning
- Deep invariant networks with differentiable augmentation layers (NeurIPS 2022 · similarity 0.87)
- Invariant Representations without Adversarial Training (NeurIPS 2018 · similarity 0.86)
- Does Invariant Graph Learning via Environment Augmentation Learn Invariance? (NeurIPS 2023 · similarity 0.86)
- Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression (ICLR 2024 · similarity 0.86)
- What Should Not Be Contrastive in Contrastive Learning (ICLR 2021 · similarity 0.85)
- How Data Augmentation Shapes Neural Representations (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
