Research map / Architectures and efficiency / RNNs and normalization
Robust Differentiable SVD
Wei Wang, Zheng Dang, Yinlin Hu, Pascal Fua, Mathieu Salzmann
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Taylor expansion instead of power iteration gradients in backward pass (ablation) | SVD-PI · CIFAR10 ZCA (d=64) | mean error (%): 4.59 -> 4.50 | Improved |
| use Taylor expansion instead of gradient clipping (ablation) | SVD-Clip · CIFAR10 ZCA (d=64) | mean error (%): 4.83 -> 4.50 | Improved |
| use Taylor gradients instead of plain SVD gradients (ablation) | SVD · CIFAR10 ZCA (d=8) | success rate: 0% -> 100% | Improved |
| raise epsilon from 1e-4 to 1e-2 (K=19) (ablation) | SVD-Taylor · CIFAR100 ZCA (d=64) | success rate: 50% -> 87.5% | Improved |
| lower Taylor degree K from 19 to 9 (ablation) | SVD-Taylor · CIFAR100 ZCA (d=64) | success rate: 87.5% -> 100% | Improved |
| lower Taylor degree K from 19 to 9 (ablation) | SVD-Taylor · CIFAR100 ZCA (d=64) | mean error (%): 21.45 -> 21.14 | Improved |
| use larger matrix groups d=32 with Taylor instead of batch norm (ablation) | ResNet50 · CIFAR100 | mean error (%): 21.62 -> 19.47 | Improved |
| use Taylor instead of SVD-PI (ablation) | ResNet50 · CIFAR100 ZCA (d=4) | mean error (%): 20.98 -> 20.34 | Improved |
| use Taylor instead of SVD-PI (ablation) | ResNet18 · Tiny ImageNet val (d=64) | mean error (%): 36.83 -> 36.72 | No clear effect |
| use Taylor instead of SVD-PI (ablation) | AlexNet · ImageNet second-order pooling | top-1 error: not converged -> 38.74 | Improved |
| use SVD-WCT instead of GD-WCT regularized network (ablation) | style transfer · Artworks | mean classification accuracy: 64.71 -> 83.30 | Improved |
| use Taylor instead of gradient clipping in WCT (ablation) | style transfer · Artworks | mean classification accuracy: 60.27 -> 83.30 | Improved |
Closest papers by meaning
- Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems (TPAMI 2020 · similarity 0.87)
- Backpropagation-Friendly Eigendecomposition (NeurIPS 2019 · similarity 0.87)
- Amortized Eigendecomposition for Neural Networks (NeurIPS 2024 · similarity 0.83)
- Fast Updating Truncated SVD for Representation Learning with Sparse Matrices (ICLR 2024 · similarity 0.83)
- How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization (ICLR 2024 · similarity 0.83)
- STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
