Research map / Architectures and efficiency / RNNs and normalization
Efficient Combination of Rematerialization and Offloading for Training DNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add offloading to rematerialization (POFO vs rematerialization only) | POFO · Torchvision CNNs, simulation | overhead wrt sequential | Helped |
| add rematerialization to offloading (POFO vs offload-only) at 12GB/s | POFO · Torchvision CNNs, simulation | overhead wrt sequential | Helped |
| POFO vs rematerialization-only on real V100 runs, DenseNet-169 | POFO · DenseNet-169, real GPU | overhead | Hurt |
| POFO vs other strategies on real V100 runs, ResNet-101 | POFO · ResNet-101, real GPU | overhead | Helped |
| increase memory ratio to high values (autocapper vs POFO) | autocapper · Torchvision CNNs, simulation | overhead | No clear effect |
Closest papers by meaning
- Coop: Memory is not a Commodity (NeurIPS 2023 · similarity 0.89)
- Efficient Rematerialization for Deep Networks (NeurIPS 2019 · similarity 0.89)
- HiRemate: Hierarchical Approach for Efficient Re-materialization of Neural Networks (ICML 2025 · similarity 0.87)
- Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage (NeurIPS 2025 · similarity 0.83)
- DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing (AAAI 2024 · similarity 0.83)
- FedMef: Towards Memory-efficient Federated Dynamic Pruning (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.