AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

AdaGC: Enhancing LLM Pretraining Stability via Adaptive Gradient Clipping

Guoxia Wang, Shuai Li, Congliang Chen, Jinle Zeng, Jiabin Yang, Dianhai Yu, Yanjun Ma, Li Shen

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Neural scaling laws · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use AdaGC instead of GlobalGCLlama-2 7B · spike scorespike score (%): 0.0333 -> 0.0000Improved
use AdaGC instead of GlobalGCQwen3-1.7B (QK-Norm) · spike scorespike score (%): 0.2842 -> 0.0053Improved
use AdaGC instead of GlobalGCMixtral 8x1B · zero-shot averageaccuracy (%): 47.74 -> 49.01Improved
use AdaGC instead of GlobalGCLlama-2 7B · zero-shot averageaccuracy (%): 49.69 -> 51.01Improved
use AdaGC instead of GlobalGCLlama-2 1.3B · zero-shot averageaccuracy (%): 46.32 -> 46.33No clear effect
use AdaGC with eps 1e-15 instead of GlobalGC with eps 1e-15ERNIE 10B-A1.4B, 350B tokens · general-ability averageaverage score: 25.71 -> 28.19Improved
use AdaGC instead of GlobalGC (eps 1e-8)ERNIE 10B-A1.4B, 1T tokens · general-ability averageaverage score: 34.52 -> 34.73No clear effect
use AdaGC with Muon optimizer instead of GlobalGCLlama-2 1.3B · zero-shot averageaccuracy (%): 47.04 -> 47.18No clear effect
use AdaGC with Lion optimizer instead of GlobalGCCLIP ViT-Base · zero-shot averageaccuracy (%): 40.65 -> 40.81No clear effect
use AdaGC instead of GlobalGC for communicationLlama-2 7B · training costGPU hours: 1468.2 -> 1402.4Improved
add GlobalGC warm-up with min-norm EMA initialization (ablation)AdaGC · Llama-2 1.3B final lossfinal training loss: 2.9708 vs 2.9725 for next-bestImproved
make clipping tensor-wise instead of a single adaptive global threshold (ablation)AdaGC · Llama-2 1.3Bfinal loss: 2.9639 vs 2.9712 (0.25% higher for Global AdaGC); 1 spike vs 0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy