Research map / Architectures and efficiency / RNNs and normalization
Rethinking skip connection model as a learnable Markov chain
Dengsheng Chen, Jie Hu, Wenwen Qiang, Xiaoming Wei, Enhua Wu
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add penal connection to Transformer | Transformer · WMT-16 DE/EN and EN/DE, Q and S | PPL: 33.3 -> 7.9; 17.3 -> 11.3; 31.8 -> 6.3; 13.4 -> 8.5 | Improved |
| Add penal connection to Transformer | Transformer · WMT-16 DE/EN and EN/DE, Q and S | accuracy: 36.4 -> 70.0; 66.4 -> 70.4; 43.1 -> 71.8; 68.7 -> 72.8 | Improved |
| Add penal connection with τ=3×10⁻⁸ | ResNet50 · ImageNet-1K val | top-1 accuracy: 79.2 -> 79.4 | Improved |
| Add penal connection with τ=3×10⁻⁸ | ViT-S/16-224 · ImageNet-1K val | top-1 accuracy: 77.8 -> 78.1 | Improved |
| Add penal connection with τ=3×10⁻⁸ | DeIT-S/16-224 · ImageNet-1K val | top-1 accuracy: 79.9 -> 80.1 | Improved |
| Add penal connection with τ=3×10⁻⁸ | Swin-S/4-7-224 · ImageNet-1K val | top-1 accuracy: 83.3 -> 83.5 | Improved |
| Vary penal connection strength τ (ablation) | ImageNet classification architectures · ImageNet-1K val | top-1 accuracy: ResNet50: 79.0, 79.4, 79.2, 79.3; ViT-S/16-224: 78.0, 78.1, 78.0, 77.8; DeIT-S/16-224: 79.9, 80.1, 80.0, 79.8; Swin-S/4- | Mixed |
| Add penal connection to ResNet | ResNet · CIFAR10 and CIFAR100 at tested depths | accuracy: CIFAR10: 92.1 -> 92.3; 92.6 -> 93.2; 93.2 -> 93.4; 92.2 -> 93.4. CIFAR100: 76.3 -> 76.7; 77.6 -> 78.0; 78.5 -> 79.1 | Improved |
| Add penal connection to ResNet | ResNet101 and ResNet152 · ImageNet-1K val | accuracy: 80.7 -> 81.1; 81.1 -> 81.4 | Improved |
Closest papers by meaning
- On the Adversarial Transferability of Generalized “Skip Connections” (TPAMI 2026 · similarity 0.88)
- SkipNet: Learning Dynamic Routing in Convolutional Networks (ECCV 2018 · similarity 0.87)
- The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling (ICML 2024 · similarity 0.86)
- Lambda-Skip Connections: the architectural component that prevents Rank Collapse (ICLR 2025 · similarity 0.85)
- Beyond Skip Connection: Pooling and Unpooling Design for Elimination Singularities (AAAI 2025 · similarity 0.84)
- Deep Learning for Computing Convergence Rates of Markov Chains (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
