Research map / Label-efficient and robust learning / Recognition and long-tail
Regularizing activations in neural networks via distribution matching with the Wasserstein metric
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PER activation distribution matching | ResNet-56 · CIFAR-10 | test error: 7.21 -> 6.72 | Helped |
| Add PER activation distribution matching | ResNet-110 · CIFAR-10 | test error: 6.90 -> 6.19 | Helped |
| Use PER instead of BN | 11-layer CNN · CIFAR-100, ReLU | test error: 29.13 -> 29.29 | Hurt |
| Use PER instead of VCL | 11-layer CNN · CIFAR-10, ELU | test error: 6.26 -> 6.42 | Hurt |
| Add PER activation distribution matching | 11-layer CNN · tiny ImageNet | test error: 37.45 -> 36.74 | Helped |
| Add PER instead of L1 activation penalty | LSTM · PTB test | perplexity: 114.1 -> 114.5 | Hurt |
| Add PER instead of L1 activation penalty | LSTM · WikiText2 test | perplexity: 130.0 -> 129.6 | Helped |
| Add PER activation distribution matching | RE-LSTM · PTB test | perplexity: 108.2 -> 108.5 | Hurt |
| Add PER activation distribution matching | RE-LSTM · WikiText2 test | perplexity: 122.7 -> 117.4 | Helped |
| Add PER activation distribution matching | VD-LSTM · PTB test | perplexity: 81.1 -> 80.7 | Helped |
| Replace PER with RBN | VD-LSTM · PTB test | perplexity: 80.7 -> 86.4 | Hurt |
| Add PER activation distribution matching | VD-LSTM · WikiText2 test | perplexity: 92.9 -> 92.6 | Helped |
Closest papers by meaning
- Regularizing by the Variance of the Activations' Sample-Variances (NeurIPS 2018 · similarity 0.89)
- Towards Understanding Regularization in Batch Normalization (ICLR 2019 · similarity 0.87)
- Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel (NeurIPS 2019 · similarity 0.87)
- Network-to-Network Regularization: Enforcing Occam's Razor to Improve Generalization (NeurIPS 2021 · similarity 0.86)
- A Kernel Perspective for Regularizing Deep Neural Networks (ICML 2019 · similarity 0.86)
- Feature Learning in $L_2$-regularized DNNs: Attraction/Repulsion and Sparsity (NeurIPS 2022 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.