Research map / Architectures and efficiency / RNNs and normalization
HydraNets: Specialized Dynamic Architectures for Efficient Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Transform ResSep into HydraNet with semantic branches and dynamic gating | ResSep on ImageNet · ImageNet | Top-1 accuracy: 1.18% | Helped |
| Transform ResNet and DenseNet into HydraNet architectures, evaluating only k = 4 of nb = 20 branches | ResNet and DenseNet · CIFAR-100 | inference MADDs: 2-4× | Helped |
| Increase the total number of branches from 10 to 25 to 50 (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 61.76 -> 63.21 -> 64.40 | Helped |
| Increase dynamically executed branches from k = 1 to k = 3 or k = 5 (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 60.08 -> 63.38 -> 64.40 | Helped |
| Increase dynamically executed branches from k = 5 to k = 10 (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 64.40 -> 64.89 | Helped |
| Move the branch point from Block 3 to Block 2 (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 64.40 -> 61.35 | Hurt |
| Use clustering-based semantic class partitions instead of random partitions (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 63.07 -> 64.40 | Helped |
| Learn the partition jointly instead of using clustering-based class partitions (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 64.40 -> 61.50 | Hurt |
| Add a branch-utilization balance loss to jointly learned partitioning (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 61.50 -> 63.17 | Helped |
| Remove gating and use outputs from all branches (ablation) | Hydra-ResSep-A · ImageNet | Top-1 accuracy: 64.40 -> 66.21 | Helped |
Closest papers by meaning
- Adaptive Neural Networks for Efficient Inference (ICML 2017 · similarity 0.87)
- Refining Architectures of Deep Convolutional Neural Networks (CVPR 2016 · similarity 0.87)
- Growing Efficient Deep Networks by Structured Continuous Sparsification (ICLR 2021 · similarity 0.87)
- SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization (ICML 2017 · similarity 0.86)
- More Is Less: A More Complicated Network With Less Inference Complexity (CVPR 2017 · similarity 0.86)
- Principled Architecture-aware Scaling of Hyperparameters (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.