Research map / Architectures and efficiency / RNNs and normalization
Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks
Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, Tom Goldstein
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use recurrent (weight-shared) instead of feed-forward network of same effective depth | ResNet-style network · Prefix sums 40-bit, depth 48 | accuracy: 22.79 -> 35.22 | Improved |
| use recurrent instead of feed-forward network at depth 40 | ResNet-style network · Prefix sums 40-bit, depth 40 | accuracy: 22.17 -> 24.96 | Improved |
| use recurrent instead of feed-forward network at depth 20 | ResNet-style network · Mazes small-to-large | accuracy: 7.94 -> 12.66 | Improved |
| use recurrent instead of feed-forward network at depth 44 | ResNet-style network · Mazes small-to-large | accuracy: 22.53 -> 29.72 | Improved |
| run more recurrent iterations at test time on prefix sums | recurrent network · Prefix sums 40-bit | accuracy | Improved |
| run 5 more iterations at test time on mazes | recurrent network trained with 20 iterations · Large mazes | accuracy: 70 | Improved |
| use dilated filters instead of standard filters (ablation) | recurrent maze network · Mazes small-to-large | accuracy: 29.72 -> 50.50 | Improved |
| use dilated filters in feed-forward network (ablation) | feed-forward maze network · Mazes small-to-large, depth 20 | accuracy: 7.94 -> 19.73 | Improved |
| run more iterations at test time on chess puzzles | recurrent network · Chess puzzles | accuracy | Improved |
| use recurrent instead of feed-forward on chess puzzles in-distribution | ResNet-style network · Chess puzzles in-distribution | accuracy: 84 -> 92 | Improved |
| make feed-forward network much deeper (132, 264, 528 layers) | feed-forward network · Prefix sums 40-bit | accuracy: 53 | No clear effect |
| run more iterations at test time on in-distribution data | recurrent network · Prefix sums and mazes in-distribution | accuracy | No clear effect |
Closest papers by meaning
- Algorithm Development in Neural Networks: Insights from the Streaming Parity Task (ICML 2025 · similarity 0.86)
- Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics (ICLR 2025 · similarity 0.85)
- Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks (ICLR 2024 · similarity 0.85)
- Deep Equilibrium Algorithmic Reasoning (NeurIPS 2024 · similarity 0.85)
- Logical characterizations of recurrent graph neural networks with reals and floats (NeurIPS 2024 · similarity 0.84)
- NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Tasks (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
