Research map / Architectures and efficiency / CNN design
Learning Features with Parameter-Free Layers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace selected ResNet50 bottleneck spatial convolutions with max-pool operations | ResNet50 · ImageNet | GPU latency (ms): 8.7 -> 7.3 | Helped |
| Replace selected ResNet50 bottleneck spatial convolutions with max-pool operations | ResNet50 · ImageNet | Top-1 accuracy (%): 78.5 -> 77.1 | Hurt |
| Use the hybrid parameter-free bottleneck design | ResNet50 · ImageNet-C | mCE (%): 63.8 -> 57.5 | Helped |
| Use the hybrid parameter-free bottleneck design | ResNet50 · ImageNet-O | AUC (%): 51.7 -> 52.9 | Helped |
| Replace ResNet50 spatial convolutions with max-pool operations throughout all stages instead of using a hybrid arrangement (ablation) | ResNet50 · ImageNet | GPU latency (ms): 8.7 -> 6.8 | Helped |
| Replace ResNet50 spatial convolutions with max-pool operations throughout all stages instead of using a hybrid arrangement (ablation) | ResNet50 · ImageNet | Top-1 accuracy (%): 76.2 -> 72.0 | Hurt |
| Replace each 3×3 convolution with a 1×1 convolution (ablation) | ResNet50 · ImageNet | Top-1 accuracy (%): 76.2 -> 30.1 | Hurt |
| Replace each 3×3 convolution with a 3×3 depthwise convolution (ablation) | ResNet50 · ImageNet | GPU latency (ms): 8.7 -> 8.3 | Hurt |
| Use deformable max-pool rather than the vanilla max-pool operation (ablation) | Ours-R50 · ImageNet | Top-1 accuracy (%): 72.0 -> 75.3 | Helped |
| Use deformable max-pool rather than the vanilla max-pool operation (ablation) | ResNet50 · COCO val2017 | Bbox AP: 32.9 -> 33.2 | Helped |
| Replace half of ViT-S self-attention layers with max-pool operations | ViT-S · ImageNet, Vanilla | Top-1 accuracy (%): 73.9 -> 74.2 | Helped |
| Replace half of PiT-S self-attention layers with max-pool operations | PiT-S · ImageNet, 256-batch throughput | Throughput (imgs/sec): 952 -> 1000 | Helped |
Closest papers by meaning
- The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes (NeurIPS 2022 · similarity 0.86)
- Principled Architecture-aware Scaling of Hyperparameters (ICLR 2024 · similarity 0.85)
- PICNN: A Pathway towards Interpretable Convolutional Neural Networks (AAAI 2024 · similarity 0.84)
- ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks (CVPR 2024 · similarity 0.84)
- Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit (ICLR 2024 · similarity 0.84)
- Training the Untrainable: Introducing Inductive Bias via Representational Alignment (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.