Research map / Detection and segmentation / CNN features
Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace step decay with linear budget-aware decay | ResNet-18 on ImageNet · ImageNet, 1% budget | top-1 accuracy: .2039 -> .3063 | Helped |
| Replace step decay with linear budget-aware decay | Mask R-CNN on MS COCO · MS COCO object detection, 1% budget | box AP: .0486 -> .0513 | Helped |
| Replace step decay with linear budget-aware decay | PSPNet on Cityscapes · Cityscapes, full budget | mIoU: .7651 -> .7633 | No clear effect |
| Convert exponential and step schedules to budget-aware schedules using BAC (ablation) | ResNet-18 on CIFAR-10 · CIFAR-10, 1% budget | accuracy: exp .99 .5848 -> BAC .6086; step-d1 .5710 -> BAC .5880 | Helped |
| Use linear decay instead of randomly subsampling data offline | ResNet-18 on CIFAR-10 · CIFAR-10, 1% budget | accuracy: .3834 -> .5544 | Helped |
| Use linear schedule rather than step-d2 across batch sizes (ablation) | ResNet-18 on CIFAR-10 · CIFAR-10, 20% budget, batch size 1024 | accuracy: .5851 -> .7415 | Helped |
| Use linear schedule rather than step-d2 with initial learning rate 0.1 (ablation) | ResNet-18 on CIFAR-10 · CIFAR-10, full budget | accuracy: .9529 -> .9563 | Helped |
| Use smooth linear decay rather than step decay for architecture-rank prediction | Random ResNet-18 variants · CIFAR-10, 10 epochs (5% budget) | Kendall's tau: 0.6651 -> 0.7398 | Helped |
| Use linear schedule rather than poly schedule (ablation) | PSPNet on Cityscapes · Cityscapes, 1% budget | mIoU: .5476 -> .5424 | No clear effect |
Closest papers by meaning
- Learning Time/Memory-Efficient Deep Architectures With Budgeted Super Networks (CVPR 2018 · similarity 0.89)
- Stepsize anything: A unified learning rate schedule for budgeted-iteration training (NeurIPS 2025 · similarity 0.89)
- Multi-Scale Dense Networks for Resource Efficient Image Classification (ICLR 2018 · similarity 0.86)
- Training the Untrainable: Introducing Inductive Bias via Representational Alignment (NeurIPS 2025 · similarity 0.86)
- Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models (NeurIPS 2024 · similarity 0.84)
- Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation is Wasteful (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.