Research map / Architectures and efficiency / RNNs and normalization
Multi-Zone Unit for Recurrent Neural Networks
Fandong Meng, Jinchao Zhang, Yang Liu, Jie Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-zone recurrent transitions with self-attention, graph convolution, or capsules | RNNs · Penn Treebank test | BPC: 1.190 -> 1.181 | Improved |
| Use multi-zone recurrent transitions with self-attention, graph convolution, or capsules | RNNs · text8 test | BPC: 1.27 -> 1.249; 1.08 | Mixed |
| Use capsule-based multi-zone transitions | Deep transition GRU · SemEval 2014 Restaurant and Laptop, DS and HDS | accuracy: 78.25$\pm$0.51 -> 79.40$\pm$0.23; 56.17$\pm$0.72 -> 57.59$\pm$0.70; 71.01$\pm$0.73 -> 72.45$\pm$0.68; 47.03$\pm$1.15 -> | Improved |
| Use 2 zones instead of 4 (ablation) | satMZU+DT · Penn Treebank test | BPC: 1.189 -> 1.191 | Hurt |
| Use 8 or 16 zones instead of 4 (ablation) | satMZU+DT · Penn Treebank test | BPC: 1.189 -> 1.200; 1.189 -> 1.208 | Hurt |
| Vary the zone count from 4 to 2, 8, or 16 (ablation) | gcnMZU+DT · Penn Treebank test | BPC: 1.183 -> 1.187; 1.183 -> 1.189; 1.183 -> 1.185 | Mixed |
| Vary the zone count from 4 to 2, 8, or 16 (ablation) | capMZU+DT · Penn Treebank test | BPC: 1.181 -> 1.186; 1.181 -> 1.181; 1.181 -> 1.186 | Mixed |
| Add a multi-zone gate function rather than a regular linear gate (ablation) | satMZRNN · Penn Treebank test, shallow and deep | BPC: 1.236 -> 1.214; 1.207 -> 1.189 | Improved |
| Add a multi-zone transformation function rather than a regular linear transformation (ablation) | satMZRNN · Penn Treebank test, shallow and deep | BPC: 1.247 -> 1.214; 1.208 -> 1.189 | Improved |
| Add a multi-zone gate function rather than a regular linear gate (ablation) | gcnMZRNN · Penn Treebank test, shallow and deep | BPC: 1.224 -> 1.196; 1.197 -> 1.183 | Improved |
| Add a multi-zone transformation function rather than a regular linear transformation (ablation) | gcnMZRNN · Penn Treebank test, shallow and deep | BPC: 1.225 -> 1.196; 1.191 -> 1.183 | Improved |
| Add a multi-zone gate function rather than a regular linear gate (ablation) | capMZRNN · Penn Treebank test, shallow and deep | BPC: 1.219 -> 1.192; 1.190 -> 1.181 | Improved |
Closest papers by meaning
- Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling (AAAI 2018 · similarity 0.87)
- Fast-Slow Recurrent Neural Networks (NeurIPS 2017 · similarity 0.86)
- Learning and Generalization in RNNs (NeurIPS 2021 · similarity 0.86)
- State-Regularized Recurrent Neural Networks (ICML 2019 · similarity 0.86)
- On the Convergence Rate of Training Recurrent Neural Networks (NeurIPS 2019 · similarity 0.86)
- M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
