Research map / Optimization / Implicit bias and generalization
Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models
Tianxiao Cao, Kyohei Atarashi, Hisashi Kashima
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SGD with SAM | TR-ResNet-32 · CIFAR-10 80% label noise | test accuracy: 57.33 -> 67.13 | Improved |
| replace SGD with DAS | TR-ResNet-32 · CIFAR-10 80% label noise | test accuracy: 57.33 -> 60.74 | Improved |
| replace SGD with DAS | TR-ResNet-32 · CIFAR-10 60% label noise | test accuracy: 77.41 -> 77.73 | No clear effect |
| replace ADAM with SAM | Tucker · COVID tensor completion | R2 score: 0.9482 -> 0.9485 | No clear effect |
| replace ADAM with DAS | Tucker · COVID tensor completion | R2 score: 0.9482 -> 0.9484 | No clear effect |
| replace SGD with DAS | TT-ResNet-18 · ImageNet-1k fine-tuning | Top-1: 65.47 -> 66.16 | Improved |
| replace SGD with DAS | TT-ResNet-18 · ImageNet-1k fine-tuning | runtime (s): 0.254 -> 0.254 | No clear effect |
| replace SAM with DAS | LoRETTA on OPT-6.7B · SuperGLUE/generation low-data | runtime: 2× -> 1.04× | Improved |
| replace ADAM with DAS | FLoRA on RoBERTa-large · GLUE low-data avg | average score: 83.78 -> 83.95 | Improved |
| replace ADAM with SAM | FLoRA on RoBERTa-large · GLUE low-data avg | average score: 83.78 -> 83.83 | No clear effect |
| replace ADAM with SAM | LoRETTA on OPT-6.7B · SuperGLUE/generation low-data | average score: 70.25 -> 71.72 | Improved |
| replace ADAM with DAS | LoRETTA on OPT-6.7B · SuperGLUE/generation low-data | average score: 70.25 -> 71.41 | Improved |
Closest papers by meaning
- Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems (NeurIPS 2024 · similarity 0.90)
- $\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling (NeurIPS 2024 · similarity 0.89)
- Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization (NeurIPS 2024 · similarity 0.88)
- Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise (NeurIPS 2025 · similarity 0.88)
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.88)
- A Universal Class of Sharpness-Aware Minimization Algorithms (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
