Research map / Architectures and efficiency / RNNs and normalization
Improving Lipschitz-Constrained Neural Networks by Learning Activation Functions
Stanislas Ducotterd, Alexis Goujon, Pakshal Bohra, Dimitris Perdios, Sebastian Neumayer, Michael Unser
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use learnable LLS activation instead of ReLU | 1-Lip denoiser · BSD68 (sigma=5/255) | PSNR: 36.10 -> 36.86 | Improved |
| use LLS instead of AV | 1-Lip denoiser PnP · single-coil MRI (acc 4) | PSNR: 30.61 -> 31.54 | Improved |
| use LLS instead of ReLU | PnP-FBS · multi-coil MRI PD (acc 4) | PSNR: 37.21 -> 38.68 | Improved |
| use LLS instead of AV | PnP-FBS · CT (noise std 1) | PSNR: 34.19 -> 35.15 | Improved |
| use LLS instead of ReLU | 1-Lip Wasserstein GAN discriminator · MNIST | inception score: 1.88 -> 2.38 | Improved |
| use LLS instead of ReLU | 1-Lip critic · Gaussian mixtures W1 (N=10) | W1 estimate: 5.960 -> 6.486 | Improved |
| use LLS instead of GS | 1-Lip critic · Gaussian mixtures W1 (N=20) | W1 estimate: 13.251 -> 13.243 | No clear effect |
| add small TV2 regularization lambda=1e-6 to LLS (ablation) | LLS denoiser · BSD68 (sigma=5/255) | average effective linear regions: 9.24 -> 1.21 | Improved |
| add TV2 regularization lambda=1e-6 to LLS (ablation) | LLS denoiser · BSD68 (sigma=5/255) | PSNR: 36.85 -> 36.86 | No clear effect |
| increase TV2 lambda from 1e-6 to 1e-2 (ablation) | LLS denoiser · BSD68 (sigma=5/255) | PSNR: 36.86 -> 35.15 | Hurt |
| increase TV2 lambda to 1e-3 (ablation) | LLS denoiser · BSD68 (sigma=5/255) | PSNR: 36.86 -> 36.63 | Hurt |
| use new LLS implementation (SplineProj parameterization and scaling parameter) instead of old normalization (ablation) | LLS 1-Lip network · 1D function fitting | test MSE | Improved |
Closest papers by meaning
- Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activations (ICLR 2024 · similarity 0.90)
- A provable control of sensitivity of neural networks through a direct parameterization of the overall bi-Lipschitzness (NeurIPS 2024 · similarity 0.89)
- Some Fundamental Aspects about Lipschitz Continuity of Neural Networks (ICLR 2024 · similarity 0.89)
- Approximation theory for 1-Lipschitz ResNets (NeurIPS 2025 · similarity 0.89)
- Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss (ICML 2025 · similarity 0.88)
- DP-SGD Without Clipping: The Lipschitz Neural Network Way (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
