Research map / Optimization / Implicit bias and generalization
Adaptive Knowledge Driven Regularization for Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CORR-Reg instead of Dropout | MLP · MNIST | ER (%): 1.90 ± 0.03 -> 1.74 ± 0.01 | Helped |
| Use CORR-Reg instead of Dropout | LeNet · MNIST | ER (%): 0.67 ± 0.02 -> 0.59 ± 0.01 | Helped |
| Use CORR-Reg instead of Dropout | AutoEncoder · MNIST | RE: 8637.92 ± 2.58 -> 8619.99 ± 0.93 | Helped |
| Use CORR-Reg instead of Dropout | VGG · CIFAR-10 | ER (%): 6.23 ± 0.07 -> 5.77 ± 0.05 | Helped |
| Use CORR-Reg instead of Dropout | MLP · MIMIC-III | AUC (%): 86.32 ± 0.02 -> 86.92 ± 0.04 | Helped |
| Use CORR-Reg instead of Dropout | LSTM · MIMIC-III | AUC (%): 86.19 ± 0.05 -> 86.60 ± 0.02 | Helped |
| Use CORR-Reg instead of L1-Reg | MLP · Sentence Polarity | AUC (%): 83.27 ± 0.05 -> 83.55 ± 0.03 | Helped |
| Use CORR-Reg instead of Dropout | LSTM · Sentence Polarity | AUC (%): 84.06 ± 0.13 -> 84.56 ± 0.07 | Helped |
| Increase regularization hyperparameter λ to large values (ablation) | CORR-Reg · MNIST-AE and MIMIC-III-MLP | Reconstruction Error; AUC: 100 and 1000 | Hurt |
| Vary β within 0.001 to 10 (ablation) | CORR-Reg · MNIST-AE and MIMIC-III-MLP | Reconstruction Error; AUC: 0.001 and 10 | No clear effect |
Closest papers by meaning
- Learning Neural Networks with Adaptive Regularization (NeurIPS 2019 · similarity 0.88)
- Network as Regularization for Training Deep Neural Networks: Framework, Model and Performance (AAAI 2020 · similarity 0.86)
- MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal Angles (NeurIPS 2020 · similarity 0.85)
- WLD-Reg: A Data-Dependent Within-Layer Diversity Regularizer (AAAI 2023 · similarity 0.85)
- Efficient K-Shot Learning With Regularized Deep Networks (AAAI 2018 · similarity 0.85)
- TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization (ICLR 2023 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.