Research map / Architectures and efficiency / RNNs and normalization
The Surprising Computational Power of Nondeterministic Stack RNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine nondeterministic stack states with vector stack embeddings | VRNS 3-3-5 · Penn Treebank | test perplexity: 120.74 -> 120.12 | Helped |
| Use a vector RNS-RNN instead of a plain LSTM | Penn Treebank · Penn Treebank | test perplexity: 125.90 -> 120.12 | Helped |
| Use a vector stack on the more complicated Dyck language | Dyck · Dyck | cross-entropy difference | Helped |
| Use RNS 2-3 rather than vector-stack models for large-alphabet marked reversal | MarkedReversal · MarkedReversal | cross-entropy difference | Helped |
| Use nondeterministic RNS 3-3 rather than deterministic multi-stack Sup. 3-3-3 | MarkedCopy and ww' · MarkedCopy and ww' | cross-entropy difference | Helped |
| Use multiple deterministic stacks when the input supplies boundaries and extra computation steps | MarkedReverseAndCopy · MarkedReverseAndCopy | cross-entropy difference | Helped |
| Attempt unmarked copying without explicit boundaries | Stack RNNs · ww | cross-entropy difference | Hurt |
| Increase nondeterminism and stack-vector capacity together | VRNS-RNN · Dyck | cross-entropy difference | Helped |
| Use a single-state RNS model on large-alphabet MarkedReversal | RNS 1-3 · MarkedReversal | cross-entropy difference | Hurt |
| Combine vector representations with nondeterministic guessing of the midpoint | Stack RNNs · UnmarkedReversal | cross-entropy difference | No clear effect |
Closest papers by meaning
- Learning Hierarchical Structures with Differentiable Nondeterministic Stacks (ICLR 2022 · similarity 0.91)
- Memory Architectures in Recurrent Neural Network Language Models (ICLR 2018 · similarity 0.88)
- Training Neural Networks as Recognizers of Formal Languages (ICLR 2025 · similarity 0.87)
- M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling (NeurIPS 2026 · similarity 0.85)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.85)
- Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.