Research map / Detection and segmentation / CNN features
GradAug: A New Regularization Method for Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply GradAug (width sub-networks + random scale) to ResNet-50 | ResNet-50 · ImageNet | Top-1: 76.32 -> 78.79 | Helped |
| combine GradAug with CutMix | GradAug ResNet-50 · ImageNet | Top-1: 78.79 -> 79.67 | Helped |
| add soft labels from full network and smallest sub-network (ablation) | GradAug · ImageNet | Top-1: 77.4 -> 78.8 | Helped |
| add soft labels and smallest sub-network (ablation) | GradAug · CIFAR-100 WRN-28-10 | Top-1: 82.5 -> 84.0 | Helped |
| always sample smallest sub-network (ablation) | GradAug · CIFAR-100 WRN-28-10 | Top-1: 83.8 -> 84.0 | No clear effect |
| combine random width sampling with random scale (vs each alone) (ablation) | WRN-28-10 · CIFAR-100 | Top-1: +2.43% | Helped |
| use random rotation instead of random scale in sub-network transform (ablation) | GradAug · CIFAR-100 | Top-1: 83.98 -> 83.36 | Mixed |
| use scale and rotation combined instead of scale only (ablation) | GradAug · CIFAR-100 / ImageNet | Top-1: 83.98 -> 84.21 (WRN); 78.79 -> 78.66 (ResNet-50) | Mixed |
| add ShakeDrop to CutMix | CutMix on WRN-28-10 · CIFAR-100 | Top-1: 84.08 -> 81.64 | Hurt |
| add GradAug to CutMix | CutMix on WRN-28-10 · CIFAR-100 | Top-1: 84.08 -> 85.25 | Helped |
| pretrain on ImageNet with GradAug vs CutMix/Mixup, then finetune | Mask-RCNN R50 · COCO detection | box mAP: 36.5 -> 37.7 | Helped |
| apply GradAug within detection training | GradAug-pretrained Mask-RCNN · COCO detection | box mAP: 37.7 -> 38.2 | Helped |
Closest papers by meaning
- Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias (ICML 2023 · similarity 0.87)
- See Through Gradients: Image Batch Recovery via GradInversion (CVPR 2021 · similarity 0.86)
- Deconstructing the Regularization of BatchNorm (ICLR 2021 · similarity 0.86)
- DiffAug: A Diffuse-and-Denoise Augmentation for Training Robust Classifiers (NeurIPS 2024 · similarity 0.85)
- Improving Generalization and Convergence by Enhancing Implicit Regularization (NeurIPS 2024 · similarity 0.84)
- Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.