Research map / Architectures and efficiency / RNNs and normalization
Optimal Gradient Checkpoint Search for Arbitrary Computation Graphs
Jianwei Feng, Dong Huang
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the optimal checkpoint search instead of Chen's checkpoint search | GCP training · Vgg19 batch 64 | training memory (MB): 3957 -> 3565 | Improved |
| Use the optimal checkpoint search instead of Chen's checkpoint search | GCP training · Vgg16 batch 64 | training memory (MB): 3957 -> 3565 | Improved |
| Use the optimal checkpoint search instead of Chen's checkpoint search | GCP training · Vgg11 batch 64 | training memory (MB): 2781 -> 2781 | No clear effect |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · ResNet18 batch 256 | training memory (MB): 4069 -> 3677 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · ResNet34 batch 128 | training memory (MB): 2231 -> 1838 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · ResNet50 batch 64 | training memory (MB): 2714 -> 1973 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · ResNet101 batch 32 | training memory (MB): 2541 -> 1024 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · ResNet152 batch 16 | training memory (MB): 1464 -> 526 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · Densenet121 batch 32 | training memory (MB): 1629 -> 898 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · Densenet161 batch 16 | training memory (MB): 1432 -> 666 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · Densenet169 batch 32 | training memory (MB): 1774 -> 897 | Improved |
| Use the optimal checkpoint search instead of the best random checkpoint selection | GCP training · Densenet201 batch 16 | training memory (MB): 1242 -> 474 | Improved |
Closest papers by meaning
- Dynamic Tensor Rematerialization (ICLR 2021 · similarity 0.87)
- Memory Optimization for Deep Networks (ICLR 2021 · similarity 0.87)
- DOGE-Train: Discrete Optimization on GPU with End-to-End Training (AAAI 2024 · similarity 0.85)
- Accelerated training through iterative gradient propagation along the residual path (ICLR 2025 · similarity 0.84)
- Accelerating neural network training: An analysis of the AlgoPerf competition (ICLR 2025 · similarity 0.84)
- Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
