AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, Tom Goldstein

NeurIPS 2021 · Architectures and efficiency › RNNs and normalization · approach: Brain-inspired recurrent architectures · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use recurrent (weight-shared) instead of feed-forward network of same effective depthResNet-style network · Prefix sums 40-bit, depth 48accuracy: 22.79 -> 35.22Improved
use recurrent instead of feed-forward network at depth 40ResNet-style network · Prefix sums 40-bit, depth 40accuracy: 22.17 -> 24.96Improved
use recurrent instead of feed-forward network at depth 20ResNet-style network · Mazes small-to-largeaccuracy: 7.94 -> 12.66Improved
use recurrent instead of feed-forward network at depth 44ResNet-style network · Mazes small-to-largeaccuracy: 22.53 -> 29.72Improved
run more recurrent iterations at test time on prefix sumsrecurrent network · Prefix sums 40-bitaccuracyImproved
run 5 more iterations at test time on mazesrecurrent network trained with 20 iterations · Large mazesaccuracy: 70Improved
use dilated filters instead of standard filters (ablation)recurrent maze network · Mazes small-to-largeaccuracy: 29.72 -> 50.50Improved
use dilated filters in feed-forward network (ablation)feed-forward maze network · Mazes small-to-large, depth 20accuracy: 7.94 -> 19.73Improved
run more iterations at test time on chess puzzlesrecurrent network · Chess puzzlesaccuracyImproved
use recurrent instead of feed-forward on chess puzzles in-distributionResNet-style network · Chess puzzles in-distributionaccuracy: 84 -> 92Improved
make feed-forward network much deeper (132, 264, 528 layers)feed-forward network · Prefix sums 40-bitaccuracy: 53No clear effect
run more iterations at test time on in-distribution datarecurrent network · Prefix sums and mazes in-distributionaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy