Research map / Detection and segmentation / CNN features
An In-depth Study of Stochastic Backpropagation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply SBP (keep-ratio 0.5) to last 8 blocks of ViT-Tiny | ViT-Tiny · ImageNet val | Top-1 accuracy (%): 73.68 -> 73.09 | Hurt |
| Apply SBP (keep-ratio 0.5) to ViT-Tiny for memory saving | ViT-Tiny · ImageNet val | GPU memory (MB/GPU): 8248 -> 5587 | Helped |
| Apply SBP (keep-ratio 0.5) to ViT-Base | ViT-Base · ImageNet val | Top-1 accuracy (%): 81.22 -> 80.62 | Hurt |
| Apply SBP (keep-ratio 0.5) to ConvNeXt-Tiny | ConvNeXt-Tiny · ImageNet val | GPU memory (MB/GPU): 12134 -> 7059 | Helped |
| Apply SBP (keep-ratio 0.5) to ConvNeXt-Base | ConvNeXt-Base · ImageNet val | Top-1 accuracy (%): 83.8 -> 83.27 | Hurt |
| Apply SBP (keep-ratio 0.5) to ConvNeXt-Base backbone | Cascade Mask R-CNN ConvNeXt-B · COCO | GPU memory (GB/GPU): 17.4 -> 12.5 | Helped |
| Apply SBP (keep-ratio 0.5) to ConvNeXt-Base backbone | Cascade Mask R-CNN ConvNeXt-B · COCO | AP box: 52.7 -> 52.5 | No clear effect |
| Apply SBP (keep-ratio 0.5) to ConvNeXt-Tiny backbone | Mask R-CNN ConvNeXt-T · COCO | AP box: 46.2 -> 45.5 | Hurt |
| Apply SBP to 8 of 12 blocks (sweet spot) instead of all (ablation) | ViT-Tiny · ImageNet val | Top-1 accuracy: accuracy drop within 1.5% | Helped |
| Reduce keep-ratio from 0.5 to 0.25 (ablation) | ViT-Tiny SBP · ImageNet val | Top-1 accuracy: 3.04% drop | Hurt |
| Use increasing or decreasing keep-ratios instead of uniform 0.5 (ablation) | ViT-Tiny SBP · ImageNet val | Top-1 accuracy | Hurt |
| Use grid-wise instead of random mask sampling (ablation) | ViT-Tiny SBP · ImageNet val | Top-1 accuracy: 0.5% ~ 1.7% higher | Helped |
Closest papers by meaning
- Stochastic Backpropagation: A Memory Efficient Strategy for Training Video Models (CVPR 2022 · similarity 0.93)
- DeepOBS: A Deep Learning Optimizer Benchmark Suite (ICLR 2019 · similarity 0.87)
- Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation (ICML 2024 · similarity 0.86)
- Accelerated training through iterative gradient propagation along the residual path (ICLR 2025 · similarity 0.85)
- A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning (AAAI 2025 · similarity 0.84)
- SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.