Research map / Large language models / Language, safety and interpretability
Hidden Breakthroughs in Language Model Training
Sara Kangaslahti, Elan Rosenfeld, Naomi Saphra
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Cluster on POLCA decomposed loss instead of exact loss | exact-loss clustering · 3-digit addition | max carry homogeneity: 0.514 -> 0.973 | Improved |
| Cluster on POLCA instead of parameter-aligned LCA | LCA · 3-digit addition | max carry homogeneity: 0.792 -> 0.973 | Improved |
| Use POLCA instead of exact loss for hidden breakthroughs | exact-loss clustering · 3-digit addition | fraction of clusters with hidden breakthroughs: 0.0 -> 0.355 | Improved |
| Add second-order term to POLCA (ablation) | first-order POLCA · 3-digit addition | max carry homogeneity: 0.948 -> 0.973 | Improved |
| Add second-order term to POLCA (ablation) | first-order POLCA · 3-digit addition | F1 of carry cluster: 0.848 -> 0.850 | No clear effect |
| Use Hessian eigenvector basis instead of random orthonormal basis (ablation) | POLCA · 3-digit addition | fraction of clusters with hidden breakthroughs: 0.031 -> 0.355 | Improved |
| Use Hessian eigenvector basis instead of random orthonormal basis (ablation) | POLCA · 3-digit addition | max carry homogeneity: 0.902 -> 0.973 | Improved |
| Use correctly ordered Hessian basis instead of shuffled checkpoints (ablation) | POLCA · 3-digit addition | max carry homogeneity: 0.856 -> 0.973 | Improved |
| Cluster on projected Fisher information instead of POLCA (ablation) | POLCA · 3-digit addition | max carry homogeneity: 0.973 -> 0.664 | Hurt |
| Add second-order approximation at datapoint level (ablation) | POLCA · 3-digit addition | RMSE of approximation: RMSE 0.145 | No clear effect |
Closest papers by meaning
- Understanding Emergent Abilities of Language Models from the Loss Perspective (NeurIPS 2024 · similarity 0.88)
- Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs (ICLR 2024 · similarity 0.88)
- Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale (NeurIPS 2025 · similarity 0.88)
- Tracing the Representation Geometry of Language Models from Pretraining to Post-training (NeurIPS 2025 · similarity 0.88)
- PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs (ICLR 2025 · similarity 0.88)
- Emergence of Linear Truth Encodings in Language Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
