Research map / Label-efficient and robust learning / Recognition and long-tail
Rethinking Feature Distribution for Loss Functions in Image Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace softmax-based classification loss with L-GM | 6-layer CNN · MNIST test, 2-D and 100-D | error rate (%): 1.30 $\pm$ 0.02 -> 1.17 $\pm$ 0.01; 0.43 $\pm$ 0.01 -> 0.39 $\pm$ 0.01 | Helped |
| Add a positive classification margin (α=1.0 instead of α=0) (ablation) | L-GM on MNIST · MNIST test, 2-D | error rate (%): 1.44 $\pm$ 0.01 -> 1.17 $\pm$ 0.01 | Helped |
| Increase the margin from α=0 to α=1.0 (ablation) | L-GM on MNIST · MNIST test, 100-D | error rate (%): 0.49 $\pm$ 0.01 -> 0.39 $\pm$ 0.01 | Helped |
| Replace softmax with L-GM | ResNet · CIFAR-10 | error rate (%): 8.75 $\pm$ 0.04 -> 7.21 $\pm$ 0.04; 6.97 $\pm$ 0.05 -> 5.61 $\pm$ 0.02; 6.43 $\pm$ 0.04 -> 4.96 $\pm$ 0.03 | Helped |
| Increase α from 0.1 to 0.2 (ablation) | L-GM on CIFAR-100 without augmentation · CIFAR-100 | error rate (%): 23.74 $\pm$ 0.08 -> 23.04 $\pm$ 0.08 | Helped |
| Increase α from 0.2 to 0.3 (ablation) | L-GM on CIFAR-100 without augmentation · CIFAR-100 | error rate (%): 23.04 $\pm$ 0.08 -> 23.80 $\pm$ 0.06 | Hurt |
| Increase α from 0.1 to 0.3 (ablation) | L-GM on augmented CIFAR-100 · CIFAR-100+ | error rate (%): 20.94 $\pm$ 0.03 -> 20.76 $\pm$ 0.03 | Helped |
| Replace softmax with L-GM | ResNet-101 · ILSVRC2012 validation, 1-crop top-1 | error rate (%): 23.5$\pm$0.2 -> 22.7$\pm$0.2 | Helped |
| Increase α from 0.001 to 0.01 (ablation) | L-GM face verification model · LFW | accuracy (%): 99.03 $\pm$ 0.03 -> 99.20 $\pm$ 0.03 | Helped |
| Use L-GM feature likelihood instead of softmax affinity for detection | MNIST classifier · MNIST adversarial verification, FGSM ε=0.3 | equal error rate (%): 37.7\% -> 3.1\% | Helped |
| Generate adversarial examples with Targeted FGSM rather than FGSM | L-GM adversarial verification · MNIST adversarial verification, ε=0.3 | equal error rate (%): 3.1\% -> 4.3\% | Hurt |
Closest papers by meaning
- Large-Margin Softmax Loss for Convolutional Neural Networks (ICML 2016 · similarity 0.91)
- Learning Towards The Largest Margins (ICLR 2022 · similarity 0.89)
- Mis-Classified Vector Guided Softmax Loss for Face Recognition (AAAI 2020 · similarity 0.87)
- Large Margin Deep Networks for Classification (NeurIPS 2018 · similarity 0.87)
- Why Do Better Loss Functions Lead to Less Transferable Features? (NeurIPS 2021 · similarity 0.86)
- On the Learning Property of Logistic and Softmax Losses for Deep Neural Networks (AAAI 2020 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.