Research map / Learning theory and algorithms / Prediction and decision losses
Memorization Through the Lens of Sample Gradients
Deepak Ravikumar, Efstathia Soufleri, Abolfazl Hashemi, Kaushik Roy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| stop training at the peak of weight norm / sample gradient instead of last epoch | ResNet18 on CIFAR-100 · CIFAR-100 (LiRA attack) | MIA AUROC: 85.48 -> 55.98 | Improved |
| stop training at the peak of weight norm / sample gradient instead of last epoch | ResNet18 on CIFAR-100 · CIFAR-100 (curvature attack) | MIA AUROC: 85.52 -> 59.42 | Improved |
| stop training at the peak of weight norm / sample gradient instead of last epoch | ResNet18 on CIFAR-100 · CIFAR-100 | UCE: 2.1839 -> 2.0821 | Improved |
| stop training at the peak of weight norm / sample gradient instead of last epoch | ResNet18 on CIFAR-100 · CIFAR-100 | ECE: 0.1017 -> 0.1382 | Hurt |
| stop training at the peak of weight norm / sample gradient instead of last epoch | ResNet18 on CIFAR-100 · CIFAR-100 | accuracy: 0.7492 -> 0.6306 | Hurt |
| replace validation-loss early stopping with sample gradient early stopping | ResNet18 on CIFAR-100 · CIFAR-100 | accuracy: 0.6262 -> 0.6306 | No clear effect |
| stop training early and accumulate loss (SGAL) instead of full-training CSG | CSG · CIFAR-100 (Inception) | cosine similarity with memorization: 0.84 -> 0.86 | Improved |
| stop training early and accumulate loss (SGAL) instead of full-training CSG | CSG · ImageNet (ResNet50) | Pearson correlation with memorization: 0.52 -> 0.62 | Improved |
| use SGAL instead of CSG for mislabel detection | CSG · CIFAR-10 mislabel detection (30% noise) | AUROC: 0.9935 -> 0.8470 | Hurt |
| train ResNet18 instead of Inception for CSG (ablation) | CSG · CIFAR-100 | cosine similarity with memorization: 0.84 -> 0.77 | Hurt |
| use ResNet18 instead of ResNet50 for CSG (ablation) | CSG · ImageNet | cosine similarity with memorization: 0.78 -> 0.68 | Hurt |
| accumulate input gradient over training instead of using last-5-checkpoint variance (VoG) | VoG · CIFAR-100 (Inception) | cosine similarity with memorization: 0.33 -> 0.84 | Improved |
Closest papers by meaning
- Towards Memorization Estimation: Fast, Formal and Free (ICML 2025 · similarity 0.94)
- Memorization Through the Lens of Curvature of Loss Function Around Samples (ICML 2024 · similarity 0.89)
- How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.88)
- On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.87)
- The Pitfalls of Memorization: When Memorization Hurts Generalization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
