Research map / Architectures and efficiency / RNNs and normalization
Path Independent Equilibrium Models Can Better Exploit Test-Time Computation
Cem Anil, Ashwini Pokle, Kaiqu Liang, Johannes Treutlein, Yuhuai Wu, Shaojie Bai, J Zico Kolter, Roger Baker Grosse
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a path independent network instead of a non-PI network (adversarial initialization attack, maze) | weight-tied network · Maze 13x13 | Attacked accuracy (%): 0 -> 100 | Improved |
| Use a path independent network instead of a non-PI network (adversarial attack, prefix sum) | weight-tied network · Prefix sum 64 bit | Attacked accuracy (%): 0 -> 100 | Improved |
| Use a PI network instead of a non-PI network | weight-tied network · Maze 13x13 | Accuracy (%): 87.12 -> 100 | Improved |
| Use a PI network instead of a non-PI network | weight-tied network · Prefix sum 64 bit | Accuracy (%): 66.66 -> 100 | Improved |
| Train with mixed zero/Gaussian noise hidden state initialization | unrolled prefix-sum networks · Prefix sum in- and OOD validation | accuracy | Improved |
| Train with randomized forward solver depth | unrolled prefix-sum networks · Prefix sum in- and OOD validation | accuracy | Improved |
| Add fixed point alignment penalty loss | unrolled prefix-sum networks · Prefix sum | AA score and upward generalization | Hurt |
| Train DEQ with phantom gradients instead of IFT | DEQ · Maze 9x9 in-dist | Residual: 1.4 -> 11.83 | Hurt |
| Train DEQ with phantom gradients instead of IFT | DEQ · Maze 9x9 in-dist | AA score: 0.99 -> 0.96 | No clear effect |
| Train DEQ with phantom gradients instead of IFT | DEQ · Maze 25x25 OOD | Accuracy (%): 100 -> 99.88 | No clear effect |
| Use Broyden solver instead of fixed point iterations at test time | unrolled network · Maze | Accuracy (%): 99.98 -> 99.97 | No clear effect |
Closest papers by meaning
- Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning (ICML 2026 · similarity 0.86)
- Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models (NeurIPS 2025 · similarity 0.85)
- Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction (ICML 2024 · similarity 0.85)
- Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · similarity 0.85)
- Algorithm Development in Neural Networks: Insights from the Streaming Parity Task (ICML 2025 · similarity 0.85)
- Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
