Research map / Architectures and efficiency / RNNs and normalization
projUNN: efficient method for training deep networks with unitary matrices
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase gradient approximation rank k from 1 to 16 (ablation) | projUNN-T and projUNN-D · Permuted MNIST | test accuracy: 92.5 -> 92.8 (T, width 116); 97.0 -> 97.0 (D, width 512) | No clear effect |
| increase rank k from 1 to 4 or 16 (ablation) | projUNN-T · Adding task | convergence speed | No clear effect |
| use rank-one rather than higher rank approximations (ablation) | projUNN-T · Copy memory task | convergence speed | Hurt |
| increase rank k up to 16 (ablation) | projUNN-T · Learning random unitary (2048x2048) | steps to converge | Helped |
| use unitary projUNN-T instead of unconstrained weights in very deep CNN | 100-layer CNN · MNIST | test accuracy: 23.5 -> 94.6 | Helped |
| impose unitary/orthogonal constraint on convolutions | ResNet9 · CIFAR10 | accuracy: 92.26% -> 82.06% | Hurt |
| projUNN-D vs ExpRNN at width 512 | RNN · Permuted MNIST | test accuracy: 96.6 -> 97.0 | Helped |
Closest papers by meaning
- Learning Unitary Operators with Help From u(n) (AAAI 2017 · similarity 0.86)
- FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network (NeurIPS 2018 · similarity 0.86)
- Full-Capacity Unitary Recurrent Neural Networks (NeurIPS 2016 · similarity 0.86)
- Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs (ICML 2017 · similarity 0.85)
- Unitary Convolutions for Message-passing and Positional Encodings on Directed Graphs (ICML 2026 · similarity 0.85)
- Unitary Convolutions for Learning on Graphs and Groups (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.