Research map / Architectures and efficiency / RNNs and normalization
Scaling Laws for Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Represent encoder and decoder parameter counts separately rather than using total parameter count alone | Scaling-law prediction for encoder-decoder NMT · English→German test sets | cross-entropy loss fit: more than 99% | Helped |
| Fit on encoder- and decoder-scaling results and predict symmetric scaling | Bivariate encoder-decoder scaling law · English→German symmetric scaling models | cross-entropy loss fit: R^2 = 99.8% | Helped |
| Scale decoder capacity rather than encoder capacity | Encoder-decoder NMT · All reported test sets | cross-entropy loss | Helped |
| Evaluate on target-original rather than source-original test data | Scaled NMT models · Target-original and source-original test sets | reducible cross-entropy loss: factor of $2.5$; factor of $11$ | Mixed |
| Increase model size | NMT evaluated on target-original test sets · Target-original test sets | BLEU | Helped |
| Increase model size past the threshold | NMT evaluated on source-original test sets · Source-original test sets | BLEU | Hurt |
| Use encoder scaling rather than decoder scaling at similar test loss | NMT generation · Test sets with BLEU evaluation | BLEU | Helped |
| Train on back-translated target-original data and scale beyond the synthetic-data generator's capacity | Models trained on back-translated data · Test sets | test cross-entropy loss: roughly $400$M parameters | Hurt |
| Train using source-original data with machine-translated target text | Models trained on forward-translated data · Training distribution | cross-entropy loss: roughly $0.16$ | Mixed |
Closest papers by meaning
- Scaling Law with Learning Rate Annealing (NeurIPS 2025 · similarity 0.91)
- Universal One-third Time Scaling in Learning Peaked Distributions (ICML 2026 · similarity 0.90)
- A Hitchhiker's Guide to Scaling Law Estimation (ICML 2025 · similarity 0.89)
- Language models scale reliably with over-training and on downstream tasks (ICLR 2025 · similarity 0.89)
- Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules (NeurIPS 2025 · similarity 0.89)
- On the Invariance and Generality of Neural Scaling Laws (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.