Research map / Architectures and efficiency / RNNs and normalization
Shallow-Deep Networks: Understanding and Mitigating Network Overthinking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add confidence-based early exits to VGG-16 | VGG-16 on CIFAR-10 · CIFAR-10 | accuracy (%): 93.1 -> 93.4 | Helped |
| Add confidence-based early exits to VGG-16 | VGG-16 on CIFAR-100 · CIFAR-100 | accuracy (%): 70.9 -> 74.4 | Helped |
| Add confidence-based early exits to WRN-32-4 | WRN-32-4 on Tiny ImageNet · Tiny ImageNet | accuracy (%): 60.3 -> 62.8 | Helped |
| Limit average inference cost to at most 50% with confidence-based early exits | SDN-trained VGG-16 · CIFAR-10 | accuracy (%): 93.1 -> 93.2 | No clear effect |
| Use joint SDN training rather than IC-only training (ablation) | VGG-16-SDN · Tiny ImageNet | maximum internal-classifier accuracy (%): 59.3 -> 63.1 | Helped |
| Use joint SDN training rather than IC-only training (ablation) | ResNet-56-SDN · Tiny ImageNet | maximum internal-classifier accuracy (%): 50.1 -> 54.1 | Helped |
| Use joint SDN training rather than IC-only training (ablation) | WRN-32-4-SDN · Tiny ImageNet | maximum internal-classifier accuracy (%): 54.6 -> 62.2 | Helped |
| Use joint SDN training rather than IC-only training (ablation) | MobileNet-SDN · Tiny ImageNet | maximum internal-classifier accuracy (%): 58.3 -> 59.6 | Helped |
| Use confidence-based early exits on backdoor inputs | Backdoored VGG-16 on CIFAR-10 · CIFAR-10 backdoor inputs | accuracy (%): 12% to 84% | Helped |
| Use confidence-based early exits on backdoor inputs | Backdoored VGG-16 on CIFAR-10 · CIFAR-10 backdoor inputs | attack target-class rate (%): 98% down to 17% | Helped |
| Use internal-classifier disagreement as an error indicator instead of final prediction confidence | VGG-16-SDN · Tiny ImageNet | separation of correct and wrong predictions by normalized score: -0.29 vs. 0.71; confidence 0.93 vs. 0.71 | Helped |
| Allow the final network layers to continue processing after a correct internal prediction | CNNs on CIFAR-10, CIFAR-100, and Tiny ImageNet · CIFAR-10, CIFAR-100, Tiny ImageNet | average inference cost: 73%; 55%; 40% | Hurt |
Closest papers by meaning
- Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign? (NeurIPS 2023 · similarity 0.88)
- Adaptive Neural Networks for Efficient Inference (ICML 2017 · similarity 0.88)
- Deep linear networks can benignly overfit when shallow ones do (ICML 2023 · similarity 0.87)
- On Local Overfitting and Forgetting in Deep Neural Networks (AAAI 2025 · similarity 0.87)
- Simplification Is All You Need against Out-of-Distribution Overconfidence (CVPR 2025 · similarity 0.86)
- Q-SENN: Quantized Self-Explaining Neural Networks (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.