Research map / Graph learning / Graph algorithms and clustering
A Graph Theoretic Framework of Recomputation Algorithms for Memory-Efficient Backpropagation
Mitsuru Kusumoto, Takuya Inoue, Gentaro Watanabe, Takuya Akiba, Masanori Koyama
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use graph-theoretic recomputation with a memory-centric strategy instead of Chen's algorithm | ApproxDP + MC · PSPNet, U-Net, ResNet50, ResNet152, VGG19, DenseNet161, GoogLeNet | peak memory consumption: PSPNet: 4.0 GB -> 2.7 GB; U-Net: 7.4 GB -> 4.7 GB; ResNet50: 3.7 GB -> 3.4 GB; ResNet152: 2.4 GB -> 2.3 GB; VGG19: 4.7 G | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · PSPNet | peak memory consumption: 3.2 GB -> 2.7 GB | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · U-Net | peak memory consumption: 6.8 GB -> 5.0 GB | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · ResNet50 | peak memory consumption: 4.7 GB -> 3.4 GB | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · ResNet152 | peak memory consumption: 2.8 GB -> 2.3 GB | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · VGG19 | peak memory consumption: 5.5 GB -> 4.5 GB | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · DenseNet161 | peak memory consumption: 1.9 GB -> 1.6 GB | Improved |
| Add liveness analysis to ApproxDP + MC (ablation) | ApproxDP + MC · GoogLeNet | peak memory consumption: 6.0 GB -> 5.2 GB | Improved |
| Use memory-centric rather than time-centric strategy (ablation) | ApproxDP · PSPNet, U-Net, ResNet50, ResNet152, VGG19, DenseNet161, GoogLeNet | peak memory consumption: PSPNet: 3.1 GB -> 2.7 GB; U-Net: 6.7 GB -> 5.0 GB; ResNet50: 4.4 GB -> 3.4 GB; ResNet152: 2.5 GB -> 2.3 GB; VGG19: 5.5 G | Mixed |
| Use approximate DP rather than exact DP (ablation) | DP recomputation strategy · benchmark networks | peak memory consumption | No clear effect |
| Use approximate DP rather than exact DP (ablation) | DP recomputation strategy · benchmark networks | DP algorithm runtime: within 1 sec versus more than 80 secs | Improved |
| Use ApproxDP + MC rather than Chen's algorithm without liveness analysis | recomputation strategy · PSPNet | peak memory consumption: 7.6 GB -> 3.2 GB | Improved |
Closest papers by meaning
- A fast heuristic to optimize time-space tradeoff for large models (NeurIPS 2023 · similarity 0.88)
- Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation (ICML 2024 · similarity 0.86)
- Locality-Aware Graph Rewiring in GNNs (ICLR 2024 · similarity 0.85)
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.85)
- Accelerated training through iterative gradient propagation along the residual path (ICLR 2025 · similarity 0.84)
- Recurrent Graph Neural Networks and Their Connections to Bisimulation and Logic (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
