Research map / Architectures and efficiency / RNNs and normalization
Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use abstraction-based equivalence queries instead of random sampling for counterexample generation (ablation) | L* extraction from GRU/LSTM trained on balanced parentheses · Balanced parentheses | train set accuracy: 99.98 vs 87.12 (GRU), 99.98 vs 94.19 (LSTM) | Helped |
| Use abstraction-based equivalence queries instead of random sampling (ablation) | L* extraction from GRU/LSTM trained on balanced parentheses · Balanced parentheses | max nesting depth: 8 vs 2 (GRU), 8 vs 3 (LSTM) | Helped |
| Use the method with no parameter tuning and an initial abstraction of depth 10 on small random regular languages | GRU networks trained on random 10-state DFAs · Random 10-state DFAs (length 1000 and train) | accuracy | Helped |
| Use a-priori quantization (q=2) partitioning instead of the proposed abstraction (ablation) | GRU networks trained on random 10-state DFAs · Random 10-state DFAs | extraction completeness and DFA size: often over 60,000 states; did not complete within 1000 s | Hurt |
| Use k-means clustering of network states for extraction instead of the proposed method (ablation) | GRU/LSTM trained on balanced parentheses · Balanced parentheses | learned nesting depth: only nesting depth 3 vs depth 8 for proposed | Hurt |
| Use the method to extract DFAs from RNNs trained on the 7 Tomita grammars | 2-layer GRU hidden 100 · Tomita grammars 1-7 | extraction correctness | Helped |
Closest papers by meaning
- Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces (AAAI 2020 · similarity 0.92)
- Learning Deterministic Weighted Automata with Queries and Counterexamples (NeurIPS 2019 · similarity 0.91)
- Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks (AAAI 2021 · similarity 0.90)
- State-Regularized Recurrent Neural Networks (ICML 2019 · similarity 0.89)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.84)
- M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.