AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

MLRegTest: A Benchmark for the Machine Learning of Regular Languages

Sam van der Poel, Dakotah Lambert, Kalina Kostyszyn, Tiantian Gao, Rahul Verma, Derek Andersen, Joanne Chau, Emily Peterson, Cody St. Clair, Paul Fodor, Chihiro Shibata, Jeffrey Heinz

JMLR 2024 · Architectures and efficiency › RNNs and normalization · approach: RNN training dynamics and gradients · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase training set size from 1k to 100kneural networks · MLRegTestaccuracy: 0.769 -> 0.887Improved
use long instead of short random test stringsneural networks · MLRegTestaccuracy: 0.944 -> 0.888Hurt
use adversarial instead of random short test stringsneural networks · MLRegTestaccuracy: 0.944 -> 0.781Hurt
use GRU instead of Simple RNN on long adversarial testGRU · MLRegTest long adversarialaccuracy: 0.662 -> 0.846Improved
use GRU instead of Transformer on long random testGRU · MLRegTest long randomaccuracy: 0.881 -> 0.966Improved
use GRU instead of LSTM on short adversarial testGRU · MLRegTest short adversarialaccuracy: 0.748 -> 0.845Improved
use complement class SL instead of coSLneural networks · SL vs coSLaccuracy: 0.862 vs 0.855No clear effect
use complement class SP instead of coSPneural networks · SP vs coSPaccuracy: 0.816 vs 0.813No clear effect
use complement class TSL instead of TcoSLneural networks · TSL vs TcoSLaccuracy: 0.839 vs 0.832No clear effect
move from Strictly Piecewise or Zp languages to Star-Free languagesneural networks · MLRegTest language classesaccuracy: 0.770 -> 0.884Improved
use general regular languages instead of SFneural networks · MLRegTest language classesaccuracy: 0.884 -> 0.781Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy