Research map / Detection and segmentation / Weakly supervised detection and segmentation
Contrastive Deep Supervision
Linfeng Zhang, Xin Chen, Junbo Zhang, Runpei Dong, Kaisheng Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contrastive losses on intermediate layers | ResNet18 · CIFAR100 | top-1 accuracy (%): 77.45 -> 80.84 | Improved |
| add contrastive losses on intermediate layers | ResNet18 · ImageNet | top-1 accuracy (%): 69.21 -> 72.85 | Improved |
| pretrain backbone with contrastive deep supervision before detection | Faster RCNN ResNet50 · COCO2017 | AP: 37.4 -> 38.3 | Improved |
| pretrain backbone with contrastive deep supervision before detection | RetinaNet ResNet50 · COCO2017 | AP: 36.5 -> 37.3 | Improved |
| pretrain backbone with cross-entropy deep supervision (DSN) instead of standard training | Faster RCNN ResNet50 · COCO2017 | AP: 37.4 -> 37.3 | No clear effect |
| add contrastive deep supervision, training from scratch | ResNet50 · CUB200 (from scratch) | top-1 accuracy (%): 60.65 -> 64.65 | Improved |
| add contrastive deep supervision, finetuning from ImageNet | ResNet50 · CUB200 (finetuned) | top-1 accuracy (%): 78.50 -> 82.10 | Improved |
| add contrastive deep supervision to feature distillation | ResNet18 student · ImageNet | top-1 accuracy (%): 69.21 -> 73.23 | Improved |
| distill projection-head embeddings rather than backbone features (ablation) | CDS distillation · CIFAR100 | top-1 accuracy (%): 82.26 -> 83.53 | Improved |
| vary where projection heads are placed (uniform, downsampling, shallow, deep) (ablation) | CDS · CIFAR100 ResNet50 | top-1 accuracy (%): 81.23, 81.31, 81.07, 80.99 | No clear effect |
| add more projection heads up to five (ablation) | CDS · CIFAR100 | top-1 accuracy (%) | Mixed |
| use contrastive deep supervision with AutoAugment and 350 epochs versus SupCon | ResNet50 · ImageNet | top-1 accuracy (%): 78.7 -> 79.8 | Improved |
Closest papers by meaning
- Dictionary Contrastive Learning for Efficient Local Supervision without Auxiliary Networks (ICLR 2024 · similarity 0.86)
- Rethinking Weak Supervision in Helping Contrastive Learning (ICML 2023 · similarity 0.84)
- Supervised Contrastive Learning (NeurIPS 2020 · similarity 0.84)
- Understanding Contrastive Learning Requires Incorporating Inductive Biases (ICML 2022 · similarity 0.84)
- Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning (ICML 2021 · similarity 0.84)
- Generalization Analysis for Deep Contrastive Representation Learning (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
