Research map / Architectures and efficiency / Attention and transformers
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that iterative self-improvement (self-generated labels filtered by length and majority voting) lets small from-scratch transformers and pretrained Llama models generalize far beyond their training difficulty on arithmetic, string, and maze tasks without architectural changes.
Problem
Standard transformers struggle with length generalization and easy-to-hard generalization on algorithmic tasks such as addition, multiplication, string copy/reverse, and maze shortest-path, and prior fixes typically require task-specific architectural or positional-encoding changes.
Approach
Alternate between generating labels with the current model on progressively harder problems and fine-tuning on the accumulated self-generated data, using unsupervised filters (relative length filtering and majority voting across seeds) to prevent error avalanches, with a standard transformer and no positional-encoding changes.
Key results
- Starting from 1-16 digit reverse addition, self-improvement generalizes to over 100-digit addition with near-perfect accuracy without additional supervision.
- Majority voting raises 5-by-6 multiplication self-generated data accuracy from an average of 31% to 93.3%; combined filtering reaches near-perfect 9-by-9 multiplication at round 31 and 10-by-10 at round 19 with accelerated schedule.
- Forward addition with length filtering maintains over 98% accuracy up to length 70 after 60 rounds, while unfiltered training collapses.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Filter 5-by-6 multiplication labels by consensus from at least 4 of 5 models | Self-improvement on multiplication · 5-by-6 multiplication | self-generated data accuracy: 31% to 93.3% | Helped |
| Remove data filtering (ablation) | Self-improvement on multiplication · 6-by-6 multiplication, round 7 | accuracy | Hurt |
| Apply both majority voting and length filtering (ablation) | Self-improvement on multiplication · 6-by-6 multiplication | accuracy | Helped |
| Apply length filtering with threshold 2 (ablation) | Self-improvement on forward addition · Forward addition | length generalization | Helped |
| Use an accelerated schedule that samples multiple difficulty levels per round (ablation) | Self-improvement on multiplication · 10-by-10 multiplication | self-improvement rounds to perfect test performance: 41 to 19 rounds | Helped |
| Train a single model with 5× more self-improvement data instead of independent models with majority voting (ablation) | Majority-voting self-improvement · Multiplication, out-of-distribution | performance | Hurt |
| Share self-improvement data across models rather than use independently generated datasets (ablation) | Majority-voting self-improvement · Multiplication, out-of-distribution | performance | Hurt |
| Use a shared initial training seed instead of different initial seeds (ablation) | Majority-voting self-improvement · Multiplication, out-of-distribution | performance | No clear effect |
| Use majority voting instead of no filtering (ablation) | Maze-solving self-improvement · Maze solving with increasing hops | generalization accuracy | Helped |
| Use majority voting instead of no filtering (ablation) | Maze-solving self-improvement · 9-hop mazes with increasing node count | generalization accuracy | Helped |
| Use move-validity verification rather than end-validity verification alone (ablation) | Verifier-filtered maze self-improvement · Maze solving with increasing hops and nodes | self-improvement performance | Helped |
| Increase the quantity of lower-quality self-generated data (ablation) | Low-performing reverse-addition models · 11-digit and 12-digit reverse addition | accuracy | Hurt |
Surprising findings
- Self-generated OOD labels are often too short (missing reasoning steps), which motivates length-based filtering.
- Models can generalize well even while memorizing their own incorrect self-generated labels, tolerating substantial label noise after enough rounds.
Limitations
- Assumes task difficulty is quantifiable and the model can handle slightly harder instances; difficulty measures for real-world tasks remain unclear.
- The model does not generate new input problems; only labels are self-generated.
- Accelerated schedule uses test-set accuracy to pick difficulty levels, which is not available in practice; multiplication uses chain-of-thought format.
Experimental setup
Datasets: synthetic arithmetic (reverse/forward addition, multiplication) generated by the authors synthetic string copy/reverse synthetic tree-graph maze shortest-path
Benchmarks: reverse addition forward addition CoT multiplication string copy/reverse maze shortest path
Metrics: exact-match accuracy accuracy on maximum digit length (OOD)
Baselines: No filtering self-improvement Oracle verifier filtering Single-model self-improvement without majority voting
Builds on: STaR ReST NoPE LoRA
Closest papers by meaning
- Self-Improvement in Language Models: The Sharpening Mechanism (ICLR 2025 · similarity 0.88)
- Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study (ICLR 2025 · similarity 0.88)
- Extrapolation by Association: Length Generalization Transfer In Transformers (NeurIPS 2025 · similarity 0.88)
- Algorithmic Capabilities of Random Transformers (NeurIPS 2024 · similarity 0.88)
- Do Efficient Transformers Really Save Computation? (ICML 2024 · similarity 0.88)
- How Do Nonlinear Transformers Learn and Generalize in In-Context Learning? (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.