Research map / Learning theory and algorithms / Kernels and regression theory
A Kernel Perspective for Regularizing Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine the deformation penalty ∥f∥τ² with the adversarial perturbation penalty ∥f∥δ² instead of using either alone (ablation) | VGG · MNIST 300 and 1k examples | test accuracy: 94.17 / 96.99; 94.42 / 97.13 -> 95.23 / 97.66 | Helped |
| Add spectral norm projection to ∥f∥τ² + ∥f∥δ² (ablation) | VGG · MNIST 300 and 1k examples | test accuracy: 95.23 / 97.66 -> 95.20 / 97.60 | Mixed |
| Include deformed digits as training examples alongside ∥f∥τ² + ∥f∥δ² (ablation) | VGG · MNIST 300 and 1k examples | test accuracy: 95.23 / 97.66 -> 95.53 / 97.56 | Mixed |
| Add a lower-bound gradient penalty to SN projection (ablation) | VGG-11 · CIFAR10 1k, without augmentation | test accuracy: 46.70 -> 46.88 | Helped |
| Add a lower-bound gradient penalty to SN projection (ablation) | ResNet-18 · CIFAR10 1k, with augmentation | test accuracy: 47.12 -> 48.73 | Helped |
| Use ∥∇f∥² instead of the gradient penalty on the loss (ablation) | CIFAR10 classifiers · CIFAR10 1k, without augmentation | test accuracy: VGG-11: 43.88 -> 46.37; ResNet-18: 41.87 -> 44.97 | Helped |
| Combine ∥f∥δ² with spectral norm projection (ablation) | 3-layer CNN · Protein homology, no data augmentation | auROC50: 0.577 -> 0.631 | Helped |
| Combine ∥f∥δ² with spectral norm projection (ablation) | 3-layer CNN · Protein homology, data augmentation | auROC50: 0.611 -> 0.639 | Helped |
| Use spectral norm upper-bound regularization alone for adversarial robustness | VGG classifier · CIFAR10 under ℓ2-bounded adversaries | robust accuracy | Hurt |
| Combine SN constraints with a lower-bound strategy such as PGD-ℓ2 | VGG classifier · CIFAR10 under ℓ2-bounded adversaries | robustness | Helped |
| Use the adversarial perturbation penalty rather than the gradient penalty | 3-layer CNN · Protein homology detection | auROC50: 0.577 vs 0.566; 0.611 vs 0.598 | Mixed |
Closest papers by meaning
- Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization (NeurIPS 2025 · similarity 0.90)
- Adversarial Training is a Form of Data-dependent Operator Norm Regularization (NeurIPS 2020 · similarity 0.89)
- Regularizing Deep Networks Using Efficient Layerwise Adversarial Training (AAAI 2018 · similarity 0.88)
- Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines (NeurIPS 2024 · similarity 0.88)
- Generalization in Kernel Regression Under Realistic Assumptions (ICML 2024 · similarity 0.87)
- Boosting Adversarial Training via Fisher-Rao Norm-based Regularization (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.