Research map / Architectures and efficiency / CNN design
Deeply-Supervised Knowledge Synergy
Dawei Sun, Anbang Yao, Aojun Zhou, Hao Zhao
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pairwise knowledge synergy to deep supervision | ResNet-18 · ImageNet validation | Top-1 error: 30.46 -> 28.68 | Improved |
| Add pairwise knowledge synergy to deep supervision | ResNet-32 · CIFAR-100 test | error: 29.89 -> 26.81 | Improved |
| Use simple AP+2FC auxiliary classifiers instead of the designed branches (ablation) | ResNet-32 DKS · CIFAR-100 test | error: 29.89(0.26) -> 35.09(0.54) | Hurt |
| Use AP+1Conv+2FC auxiliary classifiers instead of the designed branches (ablation) | ResNet-32 DKS · CIFAR-100 test | error: 30.24(0.05) -> 32.52(0.27) | Hurt |
| Use narrow auxiliary blocks (ablation) | ResNet-32 DKS · CIFAR-100 test | error: 29.52(0.30) -> 29.18(0.28) | Improved |
| Use shallow auxiliary blocks (ablation) | ResNet-32 DKS · CIFAR-100 test | error: 29.39(0.09) -> 28.69(0.28) | Improved |
| Add a classifier at C4 to the C2+C3 arrangement (ablation) | ResNet-18 DKS · ImageNet validation | Top-1 error: 28.68 -> 29.00 | Hurt |
| Use bidirectional rather than top-down matching (ablation) | ResNet-18 DKS · ImageNet validation | Top-1 accuracy gain over baseline: +0.50% -> +2.38% | Improved |
| Use bidirectional rather than bottom-up matching (ablation) | ResNet-18 DKS · ImageNet validation | Top-1 accuracy gain over baseline: +2.22% -> +2.38% | Improved |
| Add three auxiliary classifiers rather than one (ablation) | ResNet-18 DKS · ImageNet validation | Top-1 error: 31.06 -> 28.68 | Improved |
| Add a single classifier at an earlier layer rather than a deeper layer (ablation) | ResNet-18 DKS · ImageNet validation | Top-1 error: 30.69 -> 31.89 | Improved |
| Use DKS rather than DS on noisy CIFAR-10 with 50% corrupted labels (ablation) | DenseNet-40, k=12 · CIFAR-10, 50% corrupted training labels | accuracy: 80.47% -> 83.73% | Improved |
Closest papers by meaning
- The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes (NeurIPS 2022 · similarity 0.87)
- Network of Experts for Large-Scale Image Categorization (ECCV 2016 · similarity 0.87)
- Big Self-Supervised Models are Strong Semi-Supervised Learners (NeurIPS 2020 · similarity 0.87)
- ImageNet-21K Pretraining for the Masses (NeurIPS 2021 · similarity 0.86)
- VL2Lite: Task-Specific Knowledge Distillation from Large Vision-Language Models to Lightweight Networks (CVPR 2025 · similarity 0.86)
- Deep Probabilistic Supervision for Image Classification (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
