Research map / Large language models / Language, safety and interpretability

An Empirical Study on Noisy Data and LLM Pretraining Loss Divergence

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Controlled study injecting synthetic uniform random noise into clean pretraining data shows it can cause loss divergence in dense and MoE LLMs (480M to 5.2B), with distinct activation signatures from high-learning-rate divergence.

Problem

Whether noisy web-scale pretraining data actually causes LLM pretraining loss divergence, and how it differs from divergence caused by high learning rates, remains poorly understood.

Approach

Systematically inject controlled uniform random noise drawn from restricted tokenizer sub-vocabularies (insertion vs overwriting) and measure divergence probability over 20 seeds; diagnose noisy-data divergence via max attention logit thresholds (~1800) distinct from high-LR thresholds (~4000), and show QK-layernorm eliminates noise-induced divergence.

Key results

• Injected noise causes loss divergence; smaller noise vocabularies (|V_n|=5) increase divergence probability, and content of noise tokens has near-zero correlation (r=0.125) with divergence.
• Noisy-data divergences occur at max attention logit threshold ~1800, versus ~4000 for high-LR divergences, and noisy-data runs never exceed the 4000 threshold even at 100% divergence.
• Dense and active-parameter-matched MoE models (472M, 1.3B, 2.8B) show similar noise-induced divergence rates; QK-layernorm eliminates all noise-induced divergences, including at 70% noise.

Surprising findings

• Noise restricted to a small subset of the vocabulary (|V_n|=5) causes more divergence than full-vocabulary noise, contrary to prior work finding noise has limited impact.
• Depth scaling strongly degrades stability while width scaling has limited effect; a 2.5B 35-layer model diverges in 15% of runs at only 5% noise.

Limitations

• Uses synthetic uniform random noise rather than naturally occurring web noise.
• Divergence measured within truncated horizons (2e4 steps) and a compressed LR schedule rather than full 15T-token training.
• Model sizes limited to at most 5.2B parameters.

Experimental setup

Datasets: Llama 4 pretraining data mixture subset (clean corpus)

Metrics: divergence probability (percentage of seeds diverging) maximum attention logit parameter RMS norm clean-token loss Pearson correlation

Baselines: high learning rate divergence (Wortsman et al.) clean-data training

Builds on: Maximal Update Parametrization QK-layernorm z-loss Llama 3 Llama 4 pretraining data mixture

Closest papers by meaning

• LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws (ICML 2025 · similarity 0.91)
• Improving Pretraining Data Using Perplexity Correlations (ICLR 2025 · similarity 0.91)
• Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws (NeurIPS 2026 · similarity 0.90)
• Benchmarking Optimizers for Large Language Model Pretraining (NeurIPS 2026 · similarity 0.90)
• Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining (ICLR 2025 · similarity 0.90)
• Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation (NeurIPS 2025 · similarity 0.90)