Research map / Architectures and efficiency / RNNs and normalization
Non-deep Networks
Ankit Goyal, Alexey Bochkovskiy, Jia Deng, Vladlen Koltun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add rand-augment data augmentation (ablation) | ParNet baseline · ImageNet val | top-1 accuracy: 75.02 -> 75.12 | No clear effect |
| add Skip-Squeeze-Excitation (SSE) (ablation) | ParNet baseline · ImageNet val | top-1 accuracy: 75.12 -> 76.55 | Improved |
| replace ReLU with SiLU (ablation) | ParNet with SSE · ImageNet val | top-1 accuracy: 76.55 -> 76.60 | No clear effect |
| use 3 parallel branches instead of a single stream (ablation) | ParNet-M · ImageNet val | top-1 accuracy: 75.83 -> 76.6 | Improved |
| use VGG-style RepVGG-SSE blocks instead of ResNet blocks at depth 12 (ablation) | ParNet-M · ImageNet val | top-1 accuracy: 73.52 -> 75.83 | Improved |
| add SSE to the wide ResNet12 (ablation) | ResNet12-Wide · ImageNet val | top-1 accuracy: 73.52 -> 73.91 | Improved |
| use bottleneck ResNet block at depth 14 instead of basic block at depth 12 (ablation) | ResNet-Wide · ImageNet val | top-1 accuracy: 73.52 -> 72.06 | Hurt |
| fuse 3x3 and 1x1 branches into one 3x3 conv for inference | ParNet-L · ImageNet inference speed | speed (samples/sec): 112 -> 154 | Improved |
| distribute the streams across 3 GPUs | ParNet-L · ImageNet inference latency | latency (ms): 6.50 -> 4.01 | Improved |
| longer training (200 epochs, cosine annealing) | ParNet-XL · ImageNet val | top-1 accuracy: 78.55 -> 78.97 | Improved |
| train and test at resolution 320 | ParNet-XL · ImageNet val | top-1 accuracy: 78.97 -> 80.32 | Improved |
| add 10-crop testing | ParNet-XL · ImageNet val | top-1 accuracy: 80.32 -> 80.72 | Improved |
Closest papers by meaning
- Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks (CVPR 2023 · similarity 0.84)
- High Performance Zero-Memory Overhead Direct Convolutions (ICML 2018 · similarity 0.84)
- HarDNet: A Low Memory Traffic Network (ICCV 2019 · similarity 0.84)
- Adaptive Depth Networks with Skippable Sub-Paths (NeurIPS 2024 · similarity 0.83)
- Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks (ICLR 2024 · similarity 0.83)
- $\mu$PC: Scaling Predictive Coding to 100+ Layer Networks (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
