Research map / Large language models / Reasoning and chain of thought

What Do Learning Dynamics Reveal About Generalization in LLM Mathematical Reasoning?

ICML 2025 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that a finetuned LLM's pre-memorization train accuracy (accuracy on training queries before it memorizes target solution traces) predicts test accuracy and per-example robustness, and uses it for data curation that improves sample efficiency.

Problem

Understanding how LLM finetuning learning dynamics on reasoning data determine downstream generalization, distinguishing memorization of solution traces from genuine problem-solving performance.

Approach

Defines pre-memorization train accuracy: the best masked accuracy a model reaches on a training example over training epochs before its output matches the target trace (perplexity threshold). This training-dynamics metric almost linearly predicts dataset-level test accuracy (R2 ~0.88-0.97) and per-example robustness, and it can guide data collection by prioritizing low-accuracy examples.

Key results

• Pre-memorization train accuracy predicts test accuracy with R2 of 0.94 (GSM8K Llama), 0.95 (MATH Llama), 0.97 (GSM8K Gemma), and 0.88 (MATH Gemma).
• Prioritizing low pre-memorization accuracy examples yields about 2x sample efficiency over i.i.d. scaling on GSM8K with Llama3 8B.
• On MATH levels 1-3 with DeepSeekMath 7B, the approach gives about 1.5x sample efficiency; prior metrics such as gradient variance correlate weakly (-0.72 or lower magnitude) or not at all with generalization gap.

Measured effects reported in the paper

Surprising findings

• Train accuracy does not correlate with test accuracy, while pre-memorization accuracy does, even though several runs reach near-perfect train accuracy.
• Existing generalization predictors such as ATC (correlation -0.04) correlate poorly with the generalization gap in this setting.

Limitations

• The perplexity memorization threshold p must be calibrated per task or model.
• Robustness analysis relies on manually constructed task-dependent preambles.
• Data curation experiments rely on GPT-based rephrasing to generate new examples.

Experimental setup

Datasets: GSM8K MATH

Benchmarks: GSM8K MATH

Metrics: test accuracy coefficient of determination (R2) correlation coefficient sample efficiency

Baselines: i.i.d. data scaling Instruction-Following Difficulty (IFD) heuristic difficulty curation gradient variance distance from initialization Average Thresholded Confidence

Builds on: Instruction-Following Difficulty Average Thresholded Confidence

Closest papers by meaning

• Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data (ICLR 2025 · similarity 0.92)
• Quantifying Generalization Complexity for Large Language Models (ICLR 2025 · similarity 0.90)
• Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.90)
• Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models (ICLR 2025 · similarity 0.90)
• An Analysis for Reasoning Bias of Language Models with Small Initialization (ICML 2025 · similarity 0.89)