Research map / Architectures and efficiency / RNNs and normalization
MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal Angles
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MMA regularization to VGG19-BN | VGG19-BN · CIFAR100 | TOP-1 accuracy (%): 72.08 -> 73.73 | Helped |
| Add MMA regularization to WRN-16-8 | WRN-16-8 · CIFAR100 | TOP-1 accuracy (%): 78.97 -> 79.34 | Helped |
| Add MMA regularization to DenseNet-40-12 | DenseNet-40-12 · CIFAR100 | TOP-1 accuracy (%): 73.98 -> 74.61 | Helped |
| Add MMA regularization to ResNet56 | ResNet56 · TinyImageNet | TOP-1 accuracy (%): 54.80 -> 55.24 | Helped |
| Add MMA regularization to VGG16-BN | VGG16-BN · TinyImageNet | TOP-1 accuracy (%): 62.16 -> 63.37 | Helped |
| Add MMA regularization to ArcFace classification weights | ArcFace · LFW | verification accuracy (%): 99.35 -> 99.45 | Helped |
| Add MMA regularization to ArcFace classification weights | ArcFace · CFP-FP | verification accuracy (%): 95.30 -> 95.59 | Helped |
| Add MMA regularization to ArcFace classification weights | ArcFace · AgeDB-30 | verification accuracy (%): 94.62 -> 95.15 | Helped |
| Add MMA regularization to both hidden and output layers rather than hidden layers only (ablation) | VGG19-BN · CIFAR100 | TOP-1 accuracy (%): 73.45 -> 73.73 | Helped |
| Add MMA regularization to hidden layers (ablation) | VGG19-BN · CIFAR100 | TOP-1 accuracy (%): 72.08 -> 73.45 | Helped |
| Replace MMA with orthogonal regularization (ablation) | VGG19-BN · CIFAR100 | TOP-1 accuracy (%): 73.73 -> 72.83 | Hurt |
| Replace MMA with MHE (s=2) (ablation) | VGG19-BN · CIFAR100 | TOP-1 accuracy (%): 73.73 -> 71.91 | Hurt |
Closest papers by meaning
- Learning Compact Neural Networks with Regularization (ICML 2018 · similarity 0.87)
- Regularizing CNNs with Locally Constrained Decorrelations (ICLR 2017 · similarity 0.87)
- Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee (ICLR 2020 · similarity 0.87)
- Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced (NeurIPS 2018 · similarity 0.86)
- Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization (ICLR 2022 · similarity 0.86)
- Regularizing activations in neural networks via distribution matching with the Wasserstein metric (ICLR 2020 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.