Research map / Architectures and efficiency / RNNs and normalization
[Re] End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking
Sean McLeish, Long Tran-Thanh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add recall to the recurrent network (ablation) | Deep Thinking networks · 512-bit prefix sums | accuracy: $0.00 \pm 0.00$ -> $96.19 \pm 3.73$ | Improved |
| Add progressive loss to DT-Recall (ablation) | DT-Recall · 512-bit prefix sums | accuracy: $96.19 \pm 3.73$ -> $97.12 \pm 1.88$ | Improved |
| Use progressive loss without recall (ablation) | Deep Thinking network · 512-bit prefix sums | accuracy: $0.00 \pm 0.00$ -> $11.26 \pm 6.90$ | Improved |
| Sample n randomly rather than fixing n=0 (ablation) | DT-Recall · 512-bit prefix sums | accuracy: $90.27 \pm 5.95\%$ -> $97.12 \pm 1.88\%$ | Improved |
| Add recall to the recurrent network (ablation) | Deep Thinking networks · 59 × 59 mazes | accuracy: $0.00 \pm 0.00$ -> $82.72 \pm 15.14$ | Improved |
| Add progressive loss to DT-Recall (ablation) | DT-Recall · 59 × 59 mazes | accuracy: $82.72 \pm 15.14$ -> $97.30 \pm 0.68$ | Improved |
| Add recall to the recurrent network (ablation) | Deep Thinking networks · 13 × 13 mazes | accuracy: $85.59 \pm 2.81$ -> $99.94 \pm 0.03$ | Improved |
| Add recall to the recurrent network (ablation) | Deep Thinking networks · Chess puzzles 600K–700K | accuracy: $78.81 \pm 1.04$ -> $82.12 \pm 0.69$ | Improved |
| Add progressive loss to DT-Recall (ablation) | DT-Recall · Chess puzzles 600K–700K | accuracy: $82.12 \pm 0.69$ -> $82.69 \pm 0.27$ | Improved |
| Use a runtime-penalty loss (ablation) | Deep Thinking network · 512-bit prefix sums | accuracy: $72.45 \pm 8.91\%$ | Hurt |
| Use a runtime-penalty loss (ablation) | Deep Thinking network · 59 × 59 mazes | accuracy: 0% | Hurt |
| Add progressive loss to DT-Recall (ablation) | DT-Recall · Chess puzzles 600K–700K | accuracy: 83% -> 97% | Improved |
Closest papers by meaning
- HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis (NeurIPS 2024 · similarity 0.84)
- NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Tasks (NeurIPS 2024 · similarity 0.83)
- Rule Extrapolation in Language Modeling: A Study of Compositional Generalization on OOD Prompts (NeurIPS 2024 · similarity 0.82)
- Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping (NeurIPS 2026 · similarity 0.82)
- Transformers Converge to Invariant Algorithmic Cores (NeurIPS 2026 · similarity 0.82)
- Skip a Layer or Loop It? Learning Program-of-Layers in LLMs (ICML 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
