Research map / Large language models / Language, safety and interpretability
A model of errors in transformers
Suvrat Raju, Praneeth Netrapalli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the two-parameter incomplete-gamma error formula to predict accuracy vs task complexity | Gemini 2.5 Flash, Gemini 2.5 Pro, DeepSeek R1 · list reversal, nested linear transformations, dynamic programming, Tower of Hanoi, multiplication, binary addition | accuracy vs complexity | Improved |
| vanilla addition prompt with Gemini 2.5 Pro (no forced algorithm) | error formula · Vanilla addition | fit to formula | Hurt |
| force Pro to follow a specific digit-decomposition algorithm in the prompt (ablation) | Gemini 2.5 Pro addition · Addition with algorithm | fit to formula | Improved |
| give an explicit algorithm in the prompt (pseudocode or least-significant-1 rule) (ablation) | dynamic programming and Tower of Hanoi prompts · Tower of Hanoi | accuracy collapse at large c | No clear effect |
| tag digits with polynomial monomials x^k in the prompt (polynomial multiplication) (ablation) | Gemini 2.5 Flash · Multiplication | accuracy | Improved |
| set variance exponent alpha = 1/2 instead of 1 (ablation) | error formula · Nested linear transformations | chi2 and q: q for Flash 13.9 -> 53 | Mixed |
| vary alpha away from 0.5 to 1.0 range (alpha<0.5 or alpha>1) (ablation) | error formula · All tasks, mean chi2 | mean chi2: 0.49 -> 0.71 (alpha 0.25), 0.49 -> 0.61 (alpha 2) | Hurt |
Closest papers by meaning
- Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities (NeurIPS 2025 · similarity 0.87)
- Algorithmic Capabilities of Random Transformers (NeurIPS 2024 · similarity 0.87)
- Transformers Can Do Arithmetic with the Right Embeddings (NeurIPS 2024 · similarity 0.86)
- Controlling Equational Reasoning in Large Language Models with Prompt Interventions (AAAI 2025 · similarity 0.86)
- Grokking of Implicit Reasoning in Transformers: A Mechanistic Journey to the Edge of Generalization (NeurIPS 2024 · similarity 0.85)
- Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
