Research map / Probabilistic and scientific modeling / Variational and generative sampling
Sharpness-Aware Minimization Activates the Interactive Teaching's Understanding and Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames co-teaching as an EM-style iterative estimation under noisy labels and adds a SAM-style sharpness (gradient-perturbation) exchange between two peer networks, yielding Sharpness Reduction Interactive Teaching (SRIT), evaluated on noisy-label image classification benchmarks.
Problem
Interactive teaching methods such as co-teaching lack theoretical analysis of their optimization principles and convergence, and their generalization behavior under noisy labels is poorly understood.
Approach
Interprets co-teaching's small-loss sample selection as removing high-loss regions of the loss landscape and as an EM iteration with cleanliness as latent variable; adds a second level of exchange in which each network's SAM perturbation gradient, computed on the peer's selected samples, is used for the update (SRIT).
Key results
- SRIT outperforms co-teaching in all reported settings, e.g. CIFAR100 symmetric 20% noise: 59.66% vs 50.21%.
- On MNIST symmetric 20% noise, SRIT reaches 99.42% vs 97.50% for co-teaching.
- SRCNLCU (SAM applied to CNLCU) improves over CNLCU, e.g. CIFAR100 symmetric 20%: 55.84% vs 46.27%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SAM-based sharpness reduction to co-teaching | Co-teaching (SRIT) · MNIST, symmetric noise 20% | test accuracy (%): 97.50 -> 99.42 | Helped |
| Add SAM-based sharpness reduction to co-teaching | Co-teaching (SRIT) · CIFAR10, symmetric noise 20% | test accuracy (%): 82.15 -> 85.64 | Helped |
| Add SAM-based sharpness reduction to co-teaching | Co-teaching (SRIT) · CIFAR100, symmetric noise 20% | test accuracy (%): 50.21 -> 59.66 | Helped |
| Add SAM-based sharpness reduction to co-teaching | Co-teaching (SRIT) · FMNIST, symmetric noise 20% | test accuracy (%): 91.13 -> 92.68 | Helped |
| Add SAM-based sharpness reduction to co-teaching | Co-teaching (SRIT) · SVHN, symmetric noise 20% | test accuracy (%): 91.83 -> 94.95 | Helped |
| Add SAM to CNLCU-H | CNLCU-H (SRCNLCU-H) · MNIST, symmetric noise 20% | test accuracy (%): 98.70 -> 99.16 | Helped |
| Add SAM to CNLCU-H | CNLCU-H (SRCNLCU-H) · CIFAR10, symmetric noise 20% | test accuracy (%): 83.03 -> 85.43 | Helped |
| Add SAM to CNLCU-H | CNLCU-H (SRCNLCU-H) · CIFAR100, symmetric noise 20% | test accuracy (%): 46.27 -> 55.84 | Helped |
Limitations
- Paper acknowledges SAM's additional computational cost and plans to reduce it in future work.
- Experiments limited to small image classification benchmarks with a 9-layer CNN and synthetic label noise.
- Theoretical EM convergence argument is relatively informal and guarantees only local optima.
Experimental setup
Datasets: MNIST FMNIST CIFAR10 CIFAR100 SVHN
Benchmarks: MNIST FMNIST CIFAR10 CIFAR100 SVHN
Metrics: test accuracy (average of last ten epochs, 95% confidence in
Baselines: co-teaching CNLCU
Builds on: co-teaching sharpness-aware minimization CNLCU
Closest papers by meaning
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.85)
- Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise (NeurIPS 2025 · similarity 0.85)
- A Universal Class of Sharpness-Aware Minimization Algorithms (ICML 2024 · similarity 0.85)
- Zeroth-Order Sharpness-Aware Learning with Exponential Tilting (NeurIPS 2026 · similarity 0.85)
- Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization (NeurIPS 2024 · similarity 0.84)
- $\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.