Research map / Architectures and efficiency / RNNs and normalization
Robustness Emerges Early in Training Dynamics, but Is Not Preserved
Jiangang Yang, Wenhui Shi, Lu Hu, Jing Xing, Jian Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EPS (freeze shallow layers early) | ResNet-50 · ImageNet-C | Top-1 accuracy: 39.2 -> 42.5 (+3.3) | Improved |
| add AWR (revert shallow weights, low LR) | ResNet-50 · ImageNet-C | Top-1 accuracy: 39.2 -> 43.1 (+3.9) | Improved |
| add AWR with strong augmentation | ResNet-50 · ImageNet-C | Top-1 accuracy: 39.2 -> 46.5 (+7.3) | Improved |
| add EPS or AWR to WideResNet-50 | WideResNet-50 · ImageNet-100 variants | Avg. mCE: 88.0 -> 77.2 (-10.8) EPS; 88.0 -> 76.5 (-11.5) AWR | Improved |
| add EPS or AWR to MobileViT-S | MobileViT-S · ImageNet-100 variants | Avg. mCE: 89.8 -> 89.2 (-0.6) EPS; 89.8 -> 88.5 (-1.3) AWR | Improved |
| add EPS or AWR to EfficientFormer-L1 | EfficientFormer-L1 · ImageNet-100 variants | Avg. mCE: 71.4 -> 66.7 (-4.7) EPS; 71.4 -> 70.1 (-1.3) AWR | Improved |
| add EPS or AWR to AugMix and AutoAug | ResNet-50 · ImageNet-1K variants | Avg. mCE: AutoAug 74.9 -> 73.0 (-1.9) AWR; AugMix 73.4 -> 72.4 (-1.0) AWR | Improved |
| add EPS or AWR to CutMix, Label Smoothing, Dropout | ResNet-50 · ImageNet-1K variants | Avg. mCE: Dropout 77.6 -> 75.7 (-1.9) EPS; clean IN 76.3 -> 73.9 | Mixed |
| add framework to YOLOv5 and FasterRCNN | YOLOv5 · COCO-C | mAP: 19.9 -> 24.3 (+4.4); clean COCO 39.8 -> 37.9 | Mixed |
| add framework to DeepLabV3+ | DeepLabV3+ · Cityscapes-C | mIoU: 36.2 -> 39.9 (+3.7) | Improved |
| add EPS or AWR to AdaCont online TTA | AdaCont · ImageNet-C severity 5 | Top-1 accuracy: 34.9 -> 37.5 (+2.6) AWR | Improved |
| add EPS to Tent online TTA | Tent · ImageNet-C severity 5 | Top-1 accuracy: 37.3 -> 37.4 | No clear effect |
Closest papers by meaning
- One-Time Soft Alignment Enables Resilient Learning without Weight Transport (NeurIPS 2026 · similarity 0.87)
- Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos (ICLR 2025 · similarity 0.87)
- Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency (ICML 2024 · similarity 0.87)
- Robustness Reprogramming for Representation Learning (ICLR 2025 · similarity 0.87)
- Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models (NeurIPS 2024 · similarity 0.86)
- Halt Fast! Early Stopping for Certified Robustness (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
