Research map / Architectures and efficiency / RNNs and normalization
A fast heuristic to optimize time-space tradeoff for large models
Akifumi Imanishi, Zijian Xu, Masayuki Takagi, Sixue Wang, Emilio Castillo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FastSA instead of Moccasin | recomputation planning · RL 2 graph, 80% budget | cost increase: 4.9 -> 0.0 | Improved |
| use FastSA instead of Moccasin | recomputation planning · RL 4 graph, 80% budget | cost increase: 3.4 -> 2.5 | Improved |
| use FastSA instead of Checkmate MILP | recomputation planning · RL 1 graph, 80% budget | cost increase: 2.3 -> 0.3 | Improved |
| use FastSA instead of Checkmate MILP | recomputation planning · CM 2 graph, 80% budget | cost increase: 0.3 -> 0.4 | No clear effect |
| use FastSA instead of Checkmate MILP | recomputation planning · ResNet18, 80% budget | cost increase: 1.8 -> 2.9 | Hurt |
| use FastSA instead of Checkmate MILP | recomputation planning · ResNet18, 50% budget | solver time: up to 1000x faster | Improved |
| use FastSA instead of Checkmate MILP | recomputation planning · VGG11, 80% budget | cost increase: 2.9 -> 2.9 | No clear effect |
| use FastSA instead of Checkmate LP | recomputation planning · 23 vision/text models | memory reduction: 73% | Improved |
| tighten memory budget from 50% to 25% | FastSA · 23 vision/text models | average execution time increase: 7% -> 18% | Hurt |
| add node grouping before simulated annealing (ablation) | FastSA · LLaMA, 10% memory budget | final objective value: 4.4 times larger without grouping | Improved |
| use node grouping alone without SA (ablation) | FastSA · LLaMA, 10% memory budget | peak memory: 3.5e+11 -> 7.8e+10 | Improved |
| add 70% budget setting for FastSA | FastSA · RL 4 graph | cost increase: 2.5 -> 7.4 | Hurt |
Closest papers by meaning
- Resource-Efficient Transformer Pruning for Finetuning of Large Models (CVPR 2024 · similarity 0.85)
- RepAn: Enhanced Annealing through Re-parameterization (CVPR 2024 · similarity 0.85)
- MARS: Unleashing the Power of Variance Reduction for Training Large Models (ICML 2025 · similarity 0.84)
- RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks (ICLR 2025 · similarity 0.84)
- HiRemate: Hierarchical Approach for Efficient Re-materialization of Neural Networks (ICML 2025 · similarity 0.84)
- GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
