Research map / Architectures and efficiency / CNN design
Boosting the Throughput and Accelerator Utilization of Specialized CNN Inference Beyond Increasing Batch Size
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fold f images into one input with sqrt(f)x wider channels and f x smaller batch (FoldedCNN) | NoScope specialized CNNs · NoScope N1-N4, V100 FP-16 | speedup (throughput): up to 2.5x throughput | Helped |
| Fold f images into one input with wider channels (FoldedCNN) | NoScope N1 and N2 CNNs · NoScope, V100 FP-16 | FLOPs/sec utilization: up to 2.8x | Helped |
| Fold f images into one input (FoldedCNN) | NoScope N3 and N4 CNNs · NoScope, V100 FP-16 | speedup (throughput): up to 1.76x | Helped |
| Increase folding factor f (more images per input) (ablation) | NoScope FoldedCNNs · NoScope N1-N4 | accuracy: N1 drops from -0.18 (f=2) to -0.89 (f=4) | Hurt |
| Fold f=2 on NoScope N1, N2, N3, N4 (accuracy near original) | NoScope FoldedCNN f=2 · NoScope | accuracy: -0.18 to +0.25 points | No clear effect |
| Fold f=2 on N4 NoScope CNN | NoScope N4 · NoScope, V100 | speedup: 0.90x | Hurt |
| Fold f images into one input (FoldedCNN) | Microsoft game-scraping CNNs V1-V4 · game-scraping, V100 FP-16 | throughput speedup: up to 1.75x throughput, up to 1.95x FLOPs/sec | Helped |
| Fold f images into one input (FoldedCNN) with more classes | Game-scraping CNNs V1-V4 · game-scraping (11-27 classes) | accuracy: drops of 0-1.36% (f=2), 0.46-3.45% (f=3), 1.63-3.54% (f=4) | Hurt |
| Replace FLOP-reducing EfficientNet compound scaling with FoldedCNN (f=4) (ablation) | Game-scraping CNNs V1-V4 · game-scraping, V100 | accuracy and throughput: V1: 95.27% vs 93.27% accuracy, speedup 1.75 vs 1.32 | Helped |
| Fold f=4 on ResNet-18 (general CNN, CIFAR-10) | ResNet-18 CIFAR-10 · CIFAR-10, V100 | throughput speedup: 8.1% speedup, accuracy 92.10 vs 92.98 | Helped |
| Fold f=4 on ResNet-18 (CIFAR-100) | ResNet-18 CIFAR-100 · CIFAR-100 | accuracy: 70.3 original vs 65.76 (-4.54) | Hurt |
| Use curriculum learning over classes when training FoldedCNN | Game-scraping FoldedCNNs · game-scraping | accuracy | No clear effect |
Closest papers by meaning
- Searching for Fast Model Families on Datacenter Accelerators (CVPR 2021 · similarity 0.89)
- InceptionNeXt: When Inception Meets ConvNeXt (CVPR 2024 · similarity 0.87)
- More Is Less: A More Complicated Network With Less Inference Complexity (CVPR 2017 · similarity 0.86)
- Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks (CVPR 2023 · similarity 0.86)
- Faster CNNs with Direct Sparse Convolutions and Guided Pruning (ICLR 2017 · similarity 0.86)
- HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.