Research map / Architectures and efficiency / RNNs and normalization
Neural Arithmetic Logic Units
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NALU arithmetic cells instead of a Relu6 MLP | Static arithmetic learning · Arithmetic extrapolation, including addition, multiplication, and division | relative error: a+b: 42.6 -> 0.0; a×b: 10.1 -> 0.0; a/b: 37.2 -> 0.7 | Helped |
| Use recurrent NALU instead of an LSTM | Recurrent arithmetic learning · Arithmetic extrapolation over sequences | relative error: a+b: 96.1 -> 0.0; a×b: 98.2 -> 0.0 | Helped |
| Use NAC for counting and summing digit sequences | Recurrent MNIST model · MNIST digit counting and addition, sequence length 1000 | mean absolute error: counting: 198.5 -> 3.32; addition: 8811.6 -> 57.3 | Helped |
| Append NALU rather than a linear layer to the LSTM | Number-string translation model · Number-string translation, test | MAE: 29.4 -> 0.41 | Helped |
| Replace the final affine layer with NAC | MNIST parity model · MNIST parity | test accuracy: 85.1 -> 93.1 | Helped |
| Add a bias vector to NAC (ablation) | MNIST parity model · MNIST parity | test accuracy: 93.1 -> 88.1 | Hurt |
| Replace NAC weight parameterization with sigmoid(W) (ablation) | MNIST parity model · MNIST parity | test accuracy: 93.1 -> 60.0 | Hurt |
| Replace NAC weight parameterization with tanh(W) (ablation) | MNIST parity model · MNIST parity | test accuracy: 93.1 -> 87.6 | Hurt |
| Remove the bias vector (ablation) | MNIST parity affine layer · MNIST parity | test accuracy: 88.1 -> 91.4 | Helped |
| Remove the bias and use sigmoid(W) weights (ablation) | MNIST parity model · MNIST parity | test accuracy: 93.1 -> 62.5 | Hurt |
| Remove the bias and use tanh(W) weights (ablation) | MNIST parity model · MNIST parity | test accuracy: 93.1 -> 88.7 | Hurt |
| Use NALU rather than NAC | MNIST digit counting model · MNIST digit counting, sequence length 1000 | mean absolute error: 3.32 -> 4.18 | Hurt |
Closest papers by meaning
- Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics (ICLR 2025 · similarity 0.85)
- Interpreting and Improving Large Language Models in Arithmetic Calculation (ICML 2024 · similarity 0.83)
- Transformers Can Do Arithmetic with the Right Embeddings (NeurIPS 2024 · similarity 0.82)
- KLay: Accelerating Arithmetic Circuits for Neurosymbolic AI (ICLR 2025 · similarity 0.82)
- A Fast Convoluted Story: Scaling Probabilistic Inference for Integer Arithmetics (NeurIPS 2024 · similarity 0.81)
- Learning to Add, Multiply, and Execute Algorithmic Instructions Exactly with Neural Networks (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.