Research map / Architectures and efficiency / RNNs and normalization
Fast Differentiable Matrix Square Root
Yue Song, Nicu Sebe, Wei Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MPA-Lya instead of NS iteration | matrix square root computation · single matrices from 256×256 to 1024×1024 | time (ms): 6.28 -> 5.56; 9.86 -> 7.47; 12.69 -> 11.28 | Improved |
| Use MTP-Lya instead of NS iteration | matrix square root computation · single matrices from 256×256 to 1024×1024 | time (ms): 6.28 -> 4.88; 9.86 -> 5.43; 12.69 -> 7.32 | Improved |
| Use MPA-Lya instead of NS iteration | ResNet-50 global covariance pooling · ImageNet val | time (ms), single FP+BP step: 164.43 -> 110.61 | Improved |
| Use MPA-Lya instead of NS iteration | ResNet-50 global covariance pooling · ImageNet val | top-1 accuracy (%): 77.19 -> 77.13 | No clear effect |
| Use MPA-Lya instead of NS iteration | ResNet-50 global covariance pooling · ImageNet val | top-5 accuracy (%): 93.40 -> 93.45 | No clear effect |
| Use MPA-Lya instead of NS iteration | ZCA whitening · 1×64×64 matrix | FP+BP time (ms): 2.96 -> 2.52 | Improved |
| Use MTP-Lya instead of NS iteration | ZCA whitening · 1×64×64 matrix | FP+BP time (ms): 2.96 -> 2.36 | Improved |
| Use MPA-Lya instead of NS iteration | So-ViT · batch 64 of 48×48 matrices | FP+BP time (ms): 10.38 -> 3.25 | Improved |
| Use MTP-Lya instead of NS iteration | So-ViT · batch 64 of 48×48 matrices | FP+BP time (ms): 10.38 -> 2.39 | Improved |
| Use the Huang et al. NS iteration instead of the Higham NS iteration | NS iteration · ResNet-18 on CIFAR10 and CIFAR100 | mean accuracy: 4.57±0.15 vs 4.53±0.14; 21.24±0.20 vs 21.28±0.17 | No clear effect |
| Use the Huang et al. NS iteration instead of the Higham NS iteration | NS iteration · ResNet-50 on CIFAR100 | mean accuracy: 19.39±0.30 -> 19.53±0.35 | Hurt |
| Increase Lyapunov solver iterations from 5 to 9 (ablation) | backward Lyapunov solver · 10,000 randomly sampled covariance matrices | ||B_k−I||_F estimation error: 0.3541 -> 2e-7 | Improved |
Closest papers by meaning
- Fast Differentiable Matrix Square Root and Inverse Square Root (TPAMI 2022 · similarity 0.97)
- Robust Differentiable SVD (TPAMI 2021 · similarity 0.85)
- A Sparse Resultant Based Method for Efficient Minimal Solvers (CVPR 2020 · similarity 0.84)
- Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers (ICML 2025 · similarity 0.82)
- Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective (ICML 2024 · similarity 0.80)
- Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
