Research map / Label-efficient and robust learning / Self-supervised and contrastive
Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective
Quan Cui, Bingchen Zhao, Zhao-Min Chen, Borui Zhao, Renjie Song, Boyan Zhou, Jiajun Liang, Osamu Yoshie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CTC two-stage contrastive temporal coding (ablation) | ResNet-18 CosLr · CIFAR-100 | top-1 accuracy: 79.3 -> 80.1 | Improved |
| add CTC | ResNet-50 CosLr · ImageNet | top-1 accuracy: 76.1 -> 76.4 | Improved |
| add CTC on top of AutoAugment (ablation) | ResNet-18 · CIFAR-100 | top-1 accuracy: 80.4 -> 81.2 | Improved |
| add CTC on top of AutoAugment (ablation) | ResNet-50 · ImageNet | top-1 accuracy: 76.8 -> 77.2 | Improved |
| add AutoAugment to baseline training (ablation) | ResNet-18 CosLr · CIFAR-100 | top-1 accuracy: 79.3 -> 80.4 | Improved |
| transfer CTC representations (frozen backbone) | ResNet-50 · CUB200 | top-1 accuracy: 62.5 -> 63.7 | Improved |
| transfer AA+CTC representations (unfrozen backbone) | ResNet-50 · FGVC-Aircraft | top-1 accuracy: 83.4 -> 85.6 | Improved |
| use CTC pre-training instead of CosLr | ResNet-50 · COCO detection | AP bbox: 38.2 -> 39.5 | Improved |
| use CTC instead of InfoMin self-supervised pre-training | ResNet-50 · COCO detection | AP mask 75 (AP bbox 75): 42.0 -> 42.0 | No clear effect |
| use CTC instead of MoCo v1 pre-trained on Instagram-1B | ResNet-50 · iNaturalist-18 (frozen) | top-1 accuracy: 65.8 -> 66.4 | Improved |
| raise IAS loss weight alpha from 0.1 to 0.5 (ablation) | CTC · CIFAR-100 | top-1 accuracy | Mixed |
Closest papers by meaning
- A Broad Study on the Transferability of Visual Representations With Contrastive Learning (ICCV 2021 · similarity 0.89)
- Learning Optimal Representations with the Decodable Information Bottleneck (NeurIPS 2020 · similarity 0.85)
- Data-Efficient Image Recognition with Contrastive Predictive Coding (ICML 2020 · similarity 0.85)
- Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation (ICML 2019 · similarity 0.85)
- Supervised learning through the lens of compression (NeurIPS 2016 · similarity 0.85)
- Understanding the Transferability of Representations via Task-Relatedness (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
