Research map / Architectures and efficiency / RNNs and normalization
Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Rockmate (block-wise Checkmate ILP plus Rotor sequencing) instead of Rotor for memory reduction | GPT2-large · GPT2-large training | memory for activations and computational overhead: 50% memory saved at 5% overhead vs Rotor >10% overhead at same budget | Helped |
| Use Rockmate instead of Rotor, lowering minimum feasible activation budget | GPT2-large · GPT2-large training | minimum memory budget: 440 MB vs 720 MB | Helped |
| Use Rockmate instead of Rotor on ResNet models | ResNet · ResNet training | overhead at same peak memory | No clear effect |
| Increase number of rk-Checkmate budget options from (3,3) to (30,30) (ablation) | Rockmate on GPT2-medium · GPT2-medium and ResNet-101 | Rockmate makespan | Helped |
| Increase number of budget options (3,3) to (30,30) for rk-Checkmate (ablation) | Rockmate on ResNet-101 · ResNet-101 | Rockmate makespan | No clear effect |
| Replace Checkmate with Rockmate (block-level ILP + Rotor) on GPT2 with 2-10 transformer blocks | GPT2 solving time · GPT2 2-10 transformer blocks | solving time: Checkmate exceeds 30 hours on 10-block GPT2; Rockmate time nearly constant | Helped |
| Use Rockmate instead of Rotor on GPT2 networks | GPT2 throughput at matched memory · GPT2 2-10 blocks | throughput | Helped |
| Use Rockmate with identical block detection and shared rk-Checkmate solutions (ablation) | Rockmate solving time on GPT2 · GPT2 with more identical blocks | solving time | Helped |
| Use Rockmate instead of Checkmate on GPT2 with 2-10 blocks | throughput at minimum budget · GPT2 2-10 transformer blocks | throughput overhead | No clear effect |
Closest papers by meaning
- HiRemate: Hierarchical Approach for Efficient Re-materialization of Neural Networks (ICML 2025 · similarity 0.87)
- Pfeife: Automatic Pipeline Parallelism for PyTorch (ICML 2025 · similarity 0.84)
- TorchRL: A data-driven decision-making library for PyTorch (ICLR 2024 · similarity 0.83)
- DRTriton: Large-Scale Synthetic Data Driven Reinforcement Learning for Triton Kernel Generation (NeurIPS 2026 · similarity 0.83)
- Auto-reconfiguration for Latency Minimization in CPU-based DNN Serving (ICML 2025 · similarity 0.82)
- Scaling Tractable Probabilistic Circuits: A Systems Perspective (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.