Research map / Architectures and efficiency / CNN design
Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Gram iteration instead of power iteration for spectral norm estimation | spectral norm estimation on random Gaussian matrices · Random Gaussian matrices 2000 × 1000 | error ratio: -3.27e-2 ± 3.55e-3 (PI, 10 iterations) -> 6.75e-6 (GI, 10 iterations) | Helped |
| Increase Gram iteration count from 10 to 12 (ablation) | Gram iteration spectral norm estimator · Random Gaussian matrices 2000 × 1000 | error ratio: 6.75e-6 -> 4.33e-12 | Helped |
| Use Gram iteration Lipschitz regularization instead of baseline training | ResNet18 · CIFAR-10 test | accuracy (%): 93.32 ± 0.12 -> 93.48 ± 0.08 | Helped |
| Use Gram iteration Lipschitz regularization instead of weight decay | ResNet18 · CIFAR-10 test | accuracy (%): 93.30 ± 0.19 -> 93.48 ± 0.08 | Helped |
| Fine-tune a pretrained baseline with Gram iteration regularization | ResNet18 · ImageNet-1k test | accuracy (%): 69.76 -> 70.50 ± 0.05 | Helped |
| Train with Gram iteration Lipschitz regularization | ResNet18 · ImageNet-1k test | accuracy (%): 69.76 -> 70.77 ± 0.12 | Helped |
| Use 11 rather than 10 Gram iterations (ablation) | Gram iteration spectral norm estimator · Random Gaussian matrices 2000 × 1000 | error ratio: 6.75e-6 -> 4.73e-8 | Helped |
| Use explicit differentiation instead of autodifferentiation (ablation) | Gram iteration regularization · ResNet18, input image 224 × 224 | GPU memory (MB): 9770 -> 4322 | Helped |
| Replace the Gram iteration circular-convolution bound with Singla2021 for the 512 × 256 × 1 × 1 filter | ResNet18 convolutional-layer bound · 512 × 256 × 1 × 1 filter | Lipschitz bound: 2.05 -> 2.03 | Hurt |
Closest papers by meaning
- On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory (AAAI 2021 · similarity 0.93)
- Lipschitz constant estimation of Neural Networks via sparse polynomial optimization (ICLR 2020 · similarity 0.91)
- ECLipsE: Efficient Compositional Lipschitz Constant Estimation for Deep Neural Networks (NeurIPS 2024 · similarity 0.91)
- Lipschitz regularity of deep neural networks: analysis and efficient estimation (NeurIPS 2018 · similarity 0.91)
- Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks (NeurIPS 2019 · similarity 0.90)
- Some Fundamental Aspects about Lipschitz Continuity of Neural Networks (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.