Research map / Architectures and efficiency / RNNs and normalization
SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Project layer outputs onto top SVCCA directions instead of using the full layer | Convnet · CIFAR-10 | accuracy | Helped |
| Use top SVCCA directions rather than random neuron subspaces | Convnet · CIFAR-10 | accuracy | Helped |
| Use top SVCCA directions rather than maximum-activation neuron subspaces | Convnet · CIFAR-10 | accuracy | Helped |
| Sequentially freeze lower layers during training | Convnet and ResNet · CIFAR-10 | generalization accuracy | Helped |
| Sequentially freeze lower layers during training | Convnet and ResNet · CIFAR-10 | training computation | Helped |
| Compress consecutive top layers using SVCCA bottlenecks | Convnet · CIFAR-10 | accuracy: 0.35 of their original size | No clear effect |
| Compare layer representations across training time | Convnet and ResNet · CIFAR-10 | SVCCA similarity to final representation | Mixed |
| Use DFT preprocessing to make convolutional SVCCA covariance matrices block diagonal | SVCCA for convolutional layers · Convolutional layer comparisons | computational cost | Helped |
| Compare class logits with network layers using DFT CCA | ImageNet ResNet · ImageNet classes | class sensitivity | Helped |
| Project top SVCCA directions onto a limited set of neurons | Convnet representations · CIFAR-10 | accuracy | Hurt |
Closest papers by meaning
- Three Mechanisms of Feature Learning in a Linear Network (ICLR 2025 · similarity 0.86)
- Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations (ICLR 2025 · similarity 0.86)
- When Representations Align: Universality in Representation Learning Dynamics (ICML 2024 · similarity 0.85)
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra (ICLR 2025 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.85)
- Identifying Equivalent Training Dynamics (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.