Research map / Architectures and efficiency / Neural architecture search
BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budget
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| rank mixed-block candidates by Fisher potential after one minibatch rather than use a random candidate | WRN-40-2 block substitutions · CIFAR-10 test, approximately 800K parameters | error: 4.26 -> 3.79 | Helped |
| use mixed block types selected by BlockSwap instead of a single DARTS block type | WRN-40-2 · CIFAR-10 test | error: 4.49 -> 4.21 | Helped |
| use mixed block types selected by BlockSwap instead of SNIP pruning | WRN-40-2 · CIFAR-10 test, 811.4K parameters | error: 4.13 -> 3.79 | Helped |
| use a Fisher-ranked mixed-block backbone instead of a uniform G(N) backbone | Mask R-CNN with ResNet-34 · COCO-2017 val | AP: 22.3 -> 23.4 | Helped |
| use a Fisher-ranked mixed-block student instead of the ResNet-34 teacher | ResNet-34 · ImageNet validation | top-1 error: 26.73 -> 26.24 | Helped |
| increase Fisher-potential evaluation from 1 to 10 minibatches (ablation) | mixed-block architecture ranking · CIFAR-10, 400K parameter budget | Spearman rank correlation with final error: -0.685 -> -0.667 | No clear effect |
| increase Fisher-potential evaluation from 1 to 100 minibatches (ablation) | mixed-block architecture ranking · CIFAR-10, 200K parameter budget | Spearman rank correlation with final error: -0.602 -> -0.439 | Hurt |
| increase random architecture samples from 10 to 100 (ablation) | BlockSwap at a 400K parameter budget · CIFAR-10 test | error: 4.85 -> 4.54 | Helped |
| increase random architecture samples from 100 to 1000 (ablation) | BlockSwap at a 400K parameter budget · CIFAR-10 test | error: 4.54 -> 4.21 | Helped |
| replace a stage-start or stage-end block with a cheaper G(4) block (ablation) | WRN-40-2 · CIFAR-10 test | error | Hurt |
| replace a mid-stage block with a cheaper G(4) block (ablation) | WRN-40-2 · CIFAR-10 test | error | No clear effect |
Closest papers by meaning
- Neural Epitome Search for Architecture-Agnostic Network Compression (ICLR 2020 · similarity 0.87)
- Network Recasting: A Universal Method for Network Architecture Transformation (AAAI 2019 · similarity 0.87)
- Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition (NeurIPS 2021 · similarity 0.86)
- Compute Better Spent: Replacing Dense Layers with Structured Matrices (ICML 2024 · similarity 0.86)
- Block Image Compressive Sensing with Local and Global Information Interaction (AAAI 2024 · similarity 0.85)
- Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.