Research map / Architectures and efficiency / CNN design
Regularizing CNNs with Locally Constrained Decorrelations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use local decorrelation that suppresses regularization of negatively correlated weights rather than global decorrelation (ablation) | MLP · MNIST | test error | Helped |
| Increase OrthoReg strength to γ=1 instead of using no regularization (γ=0) (ablation) | 3-hidden-layer MLP · MNIST validation | error rate: 1.74% -> 1.45% | Helped |
| Set λ near 10, corresponding to orthogonality (ablation) | OrthoReg · MNIST | classification error | Helped |
| Apply OrthoReg to all layers rather than no layers (ablation) | small CNN · MNIST | error rate: 0.92 -> 0.75 | Helped |
| Add OrthoReg to the dropout baseline | small CNN + Dropout · MNIST | error rate: 0.70 ± 0.01 -> 0.66 ± 0.03 | Helped |
| Add OrthoReg to the LSUV baseline | small CNN + LSUV · MNIST | error rate: 0.86 -> 0.79 | Helped |
| Apply OrthoReg to all layers rather than only convolution layers (ablation) | small CNN · MNIST | error rate: 0.75 -> 0.75 | No clear effect |
| Add OrthoReg | 110-Layer ResNet · CIFAR-10 | test error: 6.61 -> 6.29±0.19 | Helped |
| Add OrthoReg | Wide ResNet v2 · CIFAR-10 | test error: 3.89 -> 3.69±0.01 | Helped |
| Add OrthoReg | Wide ResNet v1 · CIFAR-10 | test error: 4.37 -> 4.32± 0.05 | Helped |
| Add OrthoReg | Wide ResNet v2 · CIFAR-100 | test error: 18.85 -> 18.56 ± 0.12 | Helped |
| Add OrthoReg | 110-Layer ResNet · CIFAR-100 | test error: 28.4 -> 28.33± 0.5 | Mixed |
Closest papers by meaning
- Can We Gain More from Orthogonality Regularizations in Training Deep Networks? (NeurIPS 2018 · similarity 0.87)
- MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal Angles (NeurIPS 2020 · similarity 0.87)
- Adaptive Regularization for Large-Scale Sparse Feature Embedding Models (ICLR 2026 · similarity 0.85)
- Learning Decorrelated Representations Efficiently Using Fast Fourier Transform (CVPR 2023 · similarity 0.85)
- Feature Learning in $L_2$-regularized DNNs: Attraction/Repulsion and Sparsity (NeurIPS 2022 · similarity 0.85)
- Improving Deep Learning Optimization through Constrained Parameter Regularization (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.