Research map / Architectures and efficiency / RNNs and normalization
MLRegTest: A Benchmark for the Machine Learning of Regular Languages
Sam van der Poel, Dakotah Lambert, Kalina Kostyszyn, Tiantian Gao, Rahul Verma, Derek Andersen, Joanne Chau, Emily Peterson, Cody St. Clair, Paul Fodor, Chihiro Shibata, Jeffrey Heinz
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase training set size from 1k to 100k | neural networks · MLRegTest | accuracy: 0.769 -> 0.887 | Improved |
| use long instead of short random test strings | neural networks · MLRegTest | accuracy: 0.944 -> 0.888 | Hurt |
| use adversarial instead of random short test strings | neural networks · MLRegTest | accuracy: 0.944 -> 0.781 | Hurt |
| use GRU instead of Simple RNN on long adversarial test | GRU · MLRegTest long adversarial | accuracy: 0.662 -> 0.846 | Improved |
| use GRU instead of Transformer on long random test | GRU · MLRegTest long random | accuracy: 0.881 -> 0.966 | Improved |
| use GRU instead of LSTM on short adversarial test | GRU · MLRegTest short adversarial | accuracy: 0.748 -> 0.845 | Improved |
| use complement class SL instead of coSL | neural networks · SL vs coSL | accuracy: 0.862 vs 0.855 | No clear effect |
| use complement class SP instead of coSP | neural networks · SP vs coSP | accuracy: 0.816 vs 0.813 | No clear effect |
| use complement class TSL instead of TcoSL | neural networks · TSL vs TcoSL | accuracy: 0.839 vs 0.832 | No clear effect |
| move from Strictly Piecewise or Zp languages to Star-Free languages | neural networks · MLRegTest language classes | accuracy: 0.770 -> 0.884 | Improved |
| use general regular languages instead of SF | neural networks · MLRegTest language classes | accuracy: 0.884 -> 0.781 | Hurt |
Closest papers by meaning
- Training Neural Networks as Recognizers of Formal Languages (ICLR 2025 · similarity 0.87)
- StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs (NeurIPS 2025 · similarity 0.84)
- Neural-Symbolic Recursive Machine for Systematic Generalization (ICLR 2024 · similarity 0.84)
- Generalized Convergence Analysis of Tsetlin Automaton Based Algorithms: A Probabilistic Approach to Concept Learning (AAAI 2025 · similarity 0.84)
- Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues (ICML 2024 · similarity 0.84)
- Incremental Sequence Classification with Temporal Consistency (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
