Research map / Architectures and efficiency / RNNs and normalization
Lipschitz Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Lipschitz RNN with Euler instead of the strongest reported baseline | Lipschitz RNN · Ordered pixel-by-pixel MNIST | accuracy: 99.1% -> 99.4% | Helped |
| Use the Lipschitz RNN with Euler instead of the strongest reported baseline | Lipschitz RNN · Permuted pixel-by-pixel MNIST | accuracy: 96.2% -> 96.3% | Helped |
| Use the Lipschitz RNN with RK2 instead of Euler | Lipschitz RNN · TIMIT test | MSE: 2.82 -> 2.76 | Helped |
| Use RK2 instead of Euler | Lipschitz RNN · Ordered pixel-by-pixel MNIST | accuracy: 99.4% -> 99.3% | No clear effect |
| Use RK2 instead of Euler | Lipschitz RNN · Permuted pixel-by-pixel MNIST | accuracy: 96.3% -> 96.2% | No clear effect |
| Add the linear hidden-state term Ah (ablation) | Lipschitz RNN · Ordered pixel-by-pixel MNIST | test accuracy | Helped |
| Add the linear hidden-state term Ah (ablation) | Lipschitz RNN · Permuted pixel-by-pixel MNIST | test accuracy | Helped |
| Increase linear-term scale alpha beyond 1.6 (ablation) | Lipschitz RNN · MNIST pixel-by-pixel tasks | test accuracy | Hurt |
| Vary beta in the symmetric-skew hidden-to-hidden matrix parameterization (ablation) | Lipschitz RNN · Ordered pixel-by-pixel MNIST | test accuracy | Mixed |
| Vary beta in the symmetric-skew hidden-to-hidden matrix parameterization (ablation) | Lipschitz RNN · Permuted pixel-by-pixel MNIST | test accuracy | Mixed |
| Replace the symmetric-skew parameterization with a skew-symmetric matrix (beta=1.0) (ablation) | Lipschitz RNN · MNIST pixel-by-pixel tasks | test accuracy | Hurt |
| Add the linear term and proposed hidden-to-hidden parameterization instead of the Neural ODE RNN form | Lipschitz RNN · MNIST adversarial perturbations | PGD accuracy: 88.5% -> 93.0% | Helped |
Closest papers by meaning
- A Dynamical System Perspective for Lipschitz Neural Networks (ICML 2022 · similarity 0.90)
- Stable Recurrent Models (ICLR 2019 · similarity 0.89)
- Some Fundamental Aspects about Lipschitz Continuity of Neural Networks (ICLR 2024 · similarity 0.88)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.88)
- M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling (NeurIPS 2026 · similarity 0.86)
- A provable control of sensitivity of neural networks through a direct parameterization of the overall bi-Lipschitzness (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.