Research map / Label-efficient and robust learning / Knowledge distillation
Network Recasting: A Universal Method for Network Architecture Transformation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Recast DenseNet-100 blocks as convolution blocks instead of basic residual blocks (ablation) | DenseNet-100 · CIFAR-100+ | error rate (%): 22.39 -> 25.60 | Hurt |
| Use sequential recasting rather than training the convolution student with KD (ablation) | DenseNet-100 · CIFAR-100+ | error rate (%): 26.52 -> 25.60 | Helped |
| Use sequential recasting rather than training the convolution student with backpropagation from scratch (ablation) | DenseNet-100 · CIFAR-100+ | error rate (%): 30.05 -> 25.60 | Helped |
| Use activation-reduction rather than parameter-reduction as the VGG-16 compression criterion (ablation) | VGG-16 · ILSVRC2012 | Top5 error (%): 9.41 -> 10.38 | Hurt |
| Use the mixed basic-and-dense architecture rather than only basic blocks (ablation) | DenseNet-121 · ILSVRC2012 | Top5 error (%): 8.25 -> 7.59 | Helped |
| Use the mixed convolution-and-bottleneck architecture rather than only convolution blocks (ablation) | ResNet-50 · ILSVRC2012 | Top5 error (%): 10.39 -> 7.71 | Helped |
| Train the mixed ResNet-50 student by sequential recasting rather than KD alone (ablation) | ResNet-50 · ILSVRC2012 | Top1 error (%): 27.00 -> 25.00 | Helped |
| Train the mixed DenseNet-121 student by sequential recasting rather than KD alone (ablation) | DenseNet-121 · ILSVRC2012 | Top5 error (%): 7.65 -> 7.59 | No clear effect |
Closest papers by meaning
- BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budget (ICLR 2020 · similarity 0.87)
- N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning (ICLR 2018 · similarity 0.86)
- Few Sample Knowledge Distillation for Efficient Network Compression (CVPR 2020 · similarity 0.86)
- Neural Epitome Search for Architecture-Agnostic Network Compression (ICLR 2020 · similarity 0.85)
- ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization (NeurIPS 2025 · similarity 0.84)
- EPSD: Early Pruning with Self-Distillation for Efficient Model Compression (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.