Research map / Architectures and efficiency / RNNs and normalization
An Efficient Training Framework for Reversible Neural Architectures
Zixuan Jiang, Keren Zhu, Mingjie Liu, Jiaqi Gu, David Z. Pan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| choose per-layer M/C modes by knapsack DP instead of all-reversible (baseline-M) | Reformer on enwik8 · enwik8, batch size 1 | relative speedup: 1.321 -> 0.949 s per sample; speedup 1.392 | Improved |
| choose per-layer M/C modes by knapsack DP instead of all-reversible (baseline-M) | Reformer on enwik8 · enwik8, batch size 3 | training time per sample (s): 1.201 -> 0.917 | Improved |
| choose per-layer M/C modes by knapsack DP instead of all-reversible (baseline-M) | Reformer on enwik8 · enwik8, batch size 10 | training time per sample (s): 1.187 -> 1.129 | Improved |
| choose per-layer M/C modes by knapsack DP instead of all-reversible (baseline-M) | RevNet-104 · ImageNet (mini-batch sweep) | relative speedup: up to 1.15x | Improved |
| choose per-layer M/C modes by knapsack DP instead of all-reversible (baseline-M) | ResNeXt-101 with Inplace ABN · ImageNet (mini-batch sweep) | relative speedup: baseline-C only 0.8-2% faster than baseline-M | Improved |
| increase mini-batch size when baseline-C is feasible | Reformer on enwik8 · enwik8 | training time per sample (s): 0.869 at batch size 2 is lowest | Hurt |
| enable reversibility (M-Mode) to allow larger batches than baseline-C | Reformer on enwik8 · enwik8 | max feasible batch size: 2 -> 10 | Improved |
Closest papers by meaning
- Memory-Efficient Reversible Spiking Neural Networks (AAAI 2024 · similarity 0.87)
- Dr2Net: Dynamic Reversible Dual-Residual Networks for Memory-Efficient Finetuning (CVPR 2024 · similarity 0.84)
- PETRA: Parallel End-to-end Training with Reversible Architectures (ICLR 2025 · similarity 0.84)
- High-Performance Temporal Reversible Spiking Neural Networks with $\mathcal{O}(L)$ Training Memory and $\mathcal{O}(1)$ Inference Cost (ICML 2024 · similarity 0.84)
- On the Duality between Gradient Transformations and Adapters (ICML 2025 · similarity 0.84)
- Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
