Research map / Large language models / Instruction and fine-tuning

EvoLM: In Search of Lost Language Model Training Dynamics

NeurIPS 2025 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a suite of over 100 LLaMA-2-style 1B and 4B models from scratch through pre-training, continued pre-training, SFT, and PPO-based RL on FineWeb-Edu/FineMath and math data, and analyzes how each stage's design choices affect upstream and downstream performance.

Problem

Downstream developers lack transparent, controlled evidence about how design choices across pre-training, continued pre-training, SFT, and RL affect final language model capabilities, and intermediate checkpoints and off-the-shelf bases confound such analyses.

Approach

Train a fully controlled model suite from scratch with complete learning-rate decay across all four training stages, varying one stage at a time, and evaluate both perplexity-style cloze accuracy and generative problem-solving with in-domain and out-of-domain tasks, including ORM scores as a proxy metric.

Key results

• 1B model pre-trained on 320BT reaches 52.49 average upstream accuracy, comparable to TinyLlama-1B trained on 2T tokens (53.23).
• Under fixed 160BT pre-training, 4B SFT+RL reaches 34.8 ID Maj@16 vs 22.5 for 1B; saturation of pre-training occurs around 80x to 160x model size.
• Adding 8BT FineWeb replay during 42BT FineMath CPT raises GSM8K-Platinum Pass@1 from 19.27 (pure FineMath) to 21.01.

Measured effects reported in the paper

Surprising findings

• Excessive pre-training can degrade OOD downstream performance after SFT and RL, and RL can underperform SFT without continued pre-training.
• Intermediate checkpoints from a longer run lag behind dedicated full-schedule runs of the same token count on both upstream and math performance.

Limitations

• Only models up to 4B parameters were studied, so trends may not generalize to larger models.
• Reasoning-centric post-training only; safety, instruction-following, tool-calling, and coding objectives are unexplored.
• Only PPO was used for RL; other RL algorithms were not compared.

Experimental setup

Datasets: FineWeb-Edu FineMath MetaMathQA OpenMathInstruct2 NuminaMath GSM8K MATH

Benchmarks: HellaSwag Winogrande PIQA OBQA ARC-Easy ARC-Challenge GSM8K-Platinum MATH CRUXEval BoardgameQA TabMWP StrategyQA

Metrics: accuracy Pass@1 Maj@16 RM@16 Pass@16 correct ratio ORM score perplexity

Baselines: TinyLlama-1B Qwen1.5-4B Qwen2.5-3B Llama 3.2 1B Qwen3 1.7B Pythia OPT

Builds on: LLaMA-2 Chinchilla scaling law PPO MetaMathQA OpenMathInstruct2

Closest papers by meaning

• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.90)
• On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.90)
• PRISM: Demystifying Retention and Interaction in Mid-Training (ICML 2026 · similarity 0.89)
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.89)
• Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples (ICML 2025 · similarity 0.89)
• EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solving (NeurIPS 2025 · similarity 0.89)