Research map / Architectures and efficiency / RNNs and normalization
Neural Arithmetic Units
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace NAC_bullet with NMU | Multiplication model · Arithmetic datasets multiplication | success rate: 31% {~}^{+10%}_{-8%} -> 98% {~}^{+1%}_{-5%} | Helped |
| Replace NALU with NMU | Multiplication model · Arithmetic datasets multiplication | success rate: 0% {~}^{+4%}_{-0%} -> 98% {~}^{+1%}_{-5%} | Helped |
| Use NAU instead of NAC_plus | Addition model · Arithmetic datasets addition | median iterations to success: 2.5 \cdot 10^{5} -> 1.8 \cdot 10^{4} | Helped |
| Replace NAC_plus with NAU | Subtraction model · Arithmetic datasets subtraction | success rate: 100% {~}^{+0%}_{-4%} -> 100% {~}^{+0%}_{-4%} | No clear effect |
| Use NMU instead of NAC_bullet | Multiplication model · 10-parameter function multiplication | success rate: 13\% {~}^{+8\%}_{-5\%} -> \mathbf{94\%} {~}^{+3\%}_{-6\%} | Helped |
| Constrain NAC_bullet weights to support multiplication only (ablation) | NAC_bullet_sigma · Multiplication task with varying hidden input-size and input-range | success rate | Helped |
| Use clamped linear weights and sparsity regularization (ablation) | NAC_bullet_NMU · Multiplication task with varying hidden input-size and input-range | success rate | Helped |
| Use the NMU multiplication formulation (ablation) | Multiplication model · Multiplication task with larger hidden input-size | success rate | Helped |
| Use NMU instead of NAC_bullet | Multiplication model · Multiplication task with negative inputs | success rate | Helped |
Closest papers by meaning
- Emergence in non-neural models: grokking modular arithmetic via average gradient outer product (ICML 2025 · similarity 0.83)
- Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics (ICLR 2025 · similarity 0.82)
- Transformers Can Do Arithmetic with the Right Embeddings (NeurIPS 2024 · similarity 0.81)
- Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks (NeurIPS 2025 · similarity 0.81)
- Interpreting and Improving Large Language Models in Arithmetic Calculation (ICML 2024 · similarity 0.81)
- Learning to Add, Multiply, and Execute Algorithmic Instructions Exactly with Neural Networks (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.