Research map / Architectures and efficiency / RNNs and normalization
Backpropagation-Friendly Eigendecomposition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the SVD eigenvectors to initialize power-iteration backpropagation instead of relying on standard SVD gradients or randomly initialized PI | ZCA whitening on ResNet18 · CIFAR-10, feature dimensions d=8 to d=64 | training success rate: 100% | Helped |
| Increase ZCA feature-group dimension from d=4 to d=16 (ablation) | Ours (ZCA), ResNet18 · CIFAR-10 | minimum error: 4.59 -> 4.40 | Helped |
| Use ZCA whitening with d=64 rather than Batch Normalization | ResNet18 · CIFAR-100 | mean error: 21.85 -> 21.39 | Helped |
| Use ZCA whitening with d=32 rather than Batch Normalization | ResNet50 · CIFAR-100 | mean error: 21.62 -> 19.54 | Helped |
| Preserve less than 90% of feature variance rather than 99% (ablation) | PCA denoising · CIFAR-10 | error: 4.63±0.11 -> 5.60±0.74 | Hurt |
| Increase preserved variance from 85% to 99% (ablation) | PCA denoising · CIFAR-10 | error: 5.60±0.74 -> 4.63±0.11 | Helped |
| Preserve 99% of variance rather than 100% (ablation) | PCA denoising · CIFAR-10 | error: 4.81±0.19 -> 4.63±0.11 | Helped |
| Retain 16 eigenvectors instead of 2 (ablation) | PCA denoising · CIFAR-10 | error: 5.09 ± 0.12 -> 4.63 ± 0.17 | Helped |
| Vary retained eigenvectors across 8 to 32 (ablation) | PCA denoising · CIFAR-10 | error: 4.64 ± 0.18; 4.63 ± 0.17; 4.65 ± 0.08 | No clear effect |
| Use 2 power iterations rather than 29 (ablation) | PCA denoising · CIFAR-10 | error: 4.63 ± 0.09 vs 4.61 ± 0.03 | No clear effect |
| Use the proposed hybrid forward-SVD/backward-PI approach rather than standard PI | PCA denoising · CIFAR-10 | training stability | Helped |
Closest papers by meaning
- Amortized Eigendecomposition for Neural Networks (NeurIPS 2024 · similarity 0.90)
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density (ICML 2019 · similarity 0.86)
- Does the Data Induce Capacity Control in Deep Learning? (ICML 2022 · similarity 0.86)
- Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias (ICML 2025 · similarity 0.85)
- STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem (NeurIPS 2025 · similarity 0.85)
- A Stable Whitening Optimizer for Efficient Neural Network Training (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.