Research map / Detection and segmentation / CNN features
An Embedded Feature Whitening Approach to Deep Neural Network Optimization
Hongwei Yong, Lei Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EFW (whitened gradient) to SGDM | SGDM, ResNet50 · CIFAR100 | test accuracy (%): 77.78 -> 80.90 | Improved |
| add EFW to AdamW | AdamW, ResNet18 · CIFAR100 | test accuracy (%): 77.23 -> 78.75 | Improved |
| add EFW to SGDM | SGDM, ResNet50 · ImageNet val | top-1 accuracy (%): 76.31 -> 77.48 | Improved |
| add EFW to SGDM | Faster-RCNN R50 · COCO val2017 | AP: 37.4 -> 39.4 | Improved |
| add EFW to AdamW | Mask-RCNN Swin-T · COCO val2017 | APb: 42.7 -> 43.4 | Improved |
| add EFW to Adam | Adam, ResNet50 · Market1501 | Rank1 (%): 94.5 -> 94.5 | No clear effect |
| vary dampening epsilon from 1e-4 to 1e-3 (default) (ablation) | W-SGDM ResNet18 · CIFAR100 | test accuracy (%): 79.05 -> 79.28 | Improved |
| increase momentum alpha from 0.5 to 0.95 (ablation) | W-SGDM ResNet18 · CIFAR100 | test accuracy (%): 79.08 -> 79.28 | Improved |
| increase momentum alpha from 0.95 to 0.999 (ablation) | W-SGDM ResNet18 · CIFAR100 | test accuracy (%): 79.28 -> 78.50 | Hurt |
| update statistics and SVD less often (Txx 5/Tsvd 50 to Txx 200/Tsvd 2000) (ablation) | W-SGDM ResNet18 · CIFAR100 | test accuracy (%) and sec/epoch: 79.40 -> 79.02; 85.50 -> 24.25 sec/epoch | Mixed |
| replace ND whitening layers with EFW | SGDM ResNet18 · CIFAR100 | test accuracy (%): 78.65 -> 79.28 | Improved |
| replace ND whitening layers with EFW | SGDM ResNet18 · CIFAR100 | sec/epoch: 65.37 -> 29.78 | Improved |
Closest papers by meaning
- Improving Generalization of Batch Whitening by Convolutional Unit Optimization (ICCV 2021 · similarity 0.87)
- Switchable Whitening for Deep Representation Learning (ICCV 2019 · similarity 0.85)
- Controllable Feature Whitening for Hyperparameter-Free Bias Mitigation (ICCV 2025 · similarity 0.85)
- A Stable Whitening Optimizer for Efficient Neural Network Training (NeurIPS 2025 · similarity 0.84)
- Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model (AAAI 2025 · similarity 0.83)
- How to Fine-Tune Vision Models with SGD (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
