Research map / Label-efficient and robust learning / Noisy labels
Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Hessian-based consistent loss reweighting combined with layer-wise distance regularization (Alg. 1) vs. baselines | fine-tuning ResNet-18 on DomainNet with correlated label noise · DomainNet six domains | test accuracy: +3.26% average over baselines | Helped |
| Alg. 1 vs. baselines under independent label noise | Sketch ResNet-18 and ResNet-101, MRPC RoBERTa-Base · independent label noise 40%/60% | test accuracy: 6.43% (ResNet-18), 6.03% (ResNet-101), 2.91% (RoBERTa) average gains | Helped |
| Remove distance-based regularization (keep only loss reweighting) (ablation) | Alg. 1 on DomainNet ResNet-18 · DomainNet correlated noise, six domains | test accuracy: e.g. Clipart 83.28 -> 77.96; Sketch 66.86 -> 64.78 | Hurt |
| Remove loss reweighting (keep only distance-based regularization) (ablation) | Alg. 1 on DomainNet ResNet-18 · DomainNet correlated noise, six domains | test accuracy: e.g. Clipart 83.28 -> 82.84; Real 92.36 -> 91.67; Sketch 66.86 -> 65.96 | Hurt |
| Hessian-based distance measure reduction by Alg. 1 vs. prior fine-tuning methods | ResNet-18 fine-tuned on Clipart with weak supervision noise · Clipart weak supervision | Hessian distance measure sum sqrt(H_i): reduces by about six times more than prior methods | Helped |
| Adding distance-based (layer-wise) regularization to fine-tuning with noisy labels (ablation) | fine-tuning under label noise · DomainNet correlated noise | test accuracy: regularization-only variant outperforms all listed baselines on most domains | Helped |
Closest papers by meaning
- Improved Regularization and Robustness for Fine-tuning in Neural Networks (NeurIPS 2021 · similarity 0.92)
- Distance-Based Regularisation of Deep Networks for Fine-Tuning (ICLR 2021 · similarity 0.92)
- Tuning Stable Rank Shrinkage: Aiming at the Overlooked Structural Risk in Fine-tuning (CVPR 2024 · similarity 0.88)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.88)
- Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability (AAAI 2024 · similarity 0.87)
- LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.