Research map / Architectures and efficiency / RNNs and normalization
Training Linear Finite-State Machines
Arash Ardakani, Amir Ardakani, Warren Gross
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FSM-based network instead of SC-based TCAD'18 | MNIST MLP · MNIST test | error rate (%): 3 -> 1.28 | Improved |
| use FSM-based network instead of SC-based TCOMP'18 | MNIST MLP · MNIST test | error rate (%): 2.05 -> 1.28 | Improved |
| use FSM-based network instead of SC-based TVLSI'17 | MNIST MLP · MNIST test | error rate (%): 2.01 -> 1.28 | Improved |
| shrink FSM-based network from 784-250-250-10 to 784-70-70-10 | FSM-based network · MNIST test | error rate (%): 1.28 -> 1.66 | Hurt |
| use stream length 128 instead of 256 versus SC baselines | FSM-based network · MNIST test | stream length: 256 -> 128 | Improved |
| use two states instead of more states (ablation) | FSM-based network · MNIST test | error rate | Improved |
| increase stream length beyond 64 (ablation) | FSM-based network · MNIST test | error rate | No clear effect |
| increase stream length and number of states (ablation) | three-layer FSM Gabor network · 2D Gabor filter | MSE | Improved |
| replace LSTM with 4-state FSM-based model | char-level language model · Penn Treebank | BPC: 1.45 -> 1.52 | Hurt |
| replace LSTM with 4-state FSM-based model | char-level language model · War & Peace | BPC: 1.83 -> 1.89 | Hurt |
| replace LSTM with 4-state FSM-based model | char-level language model · Linux Kernel | BPC: 1.85 -> 1.93 | Hurt |
| replace LSTM with FSM-based model | char-level language model · Penn Treebank | number of operations: 8.1M -> 1.1M | Improved |
Closest papers by meaning
- xLSTM: Extended Long Short-Term Memory (NeurIPS 2024 · similarity 0.85)
- Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity (ICML 2025 · similarity 0.84)
- Analog Foundation Models (NeurIPS 2025 · similarity 0.83)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.83)
- xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference (ICML 2025 · similarity 0.83)
- Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
