Research map / Architectures and efficiency / RNNs and normalization

Simplified Reversible Residual Networks

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Representation geometry and collapse

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a simplified reversible residual network formulation that avoids storing activations for backpropagation, targeting memory reduction when training large networks.

Problem

Training deep neural networks requires storing per-layer activations for backpropagation, so memory consumption becomes a bottleneck for large models.

Approach

Builds on the ResNet layer formulation to make residual layers reversible and simplified so activations can be recomputed from outputs during the backward pass instead of being stored; specific mechanism not stated in the abstract.

Experimental setup

Builds on: ResNet reversible residual networks

Closest papers by meaning

• The Reversible Residual Network: Backpropagation Without Storing Activations (NeurIPS 2017 · similarity 0.96)
• Residual Alignment: Uncovering the Mechanisms of Residual Networks (NeurIPS 2023 · similarity 0.90)
• Dr2Net: Dynamic Reversible Dual-Residual Networks for Memory-Efficient Finetuning (CVPR 2024 · similarity 0.90)
• Reversible Architectures for Arbitrarily Deep Residual Neural Networks (AAAI 2018 · similarity 0.90)
• Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.87)
• PETRA: Parallel End-to-end Training with Reversible Architectures (ICLR 2025 · similarity 0.86)