Research map / Architectures and efficiency / RNNs and normalization
Decoupling Dynamical Richness from Representation Learning: Towards Practical Measurement
Yoonsoo Nam, Nayara Fonseca, Seok Hyeong Lee, Chris Mingard, Niclas Alexander Göring, Ouns El Harzli, Abdurrahman Hadi Erturk, Soufiane Hayou, Ard A. Louis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add batch normalization to VGG-16 (ablation) | VGG-16 · CIFAR-100 | test accuracy: 21.7% -> 72.0% | Improved |
| add batch normalization to VGG-16 (ablation) | VGG-16 · CIFAR-100 | D_LR: 0.66 -> 0.073 | Improved |
| increase weight decay from 0 to 1e-3 (ablation) | ResNet18 · CIFAR-10 | test accuracy: 93.5% -> 94.8% | Improved |
| increase weight decay from 0 to 1e-3 (ablation) | ResNet18 · CIFAR-10 | D_LR: 0.05 -> 0.003 | Improved |
| raise learning rate from 0.005 to 0.05 (ablation) | ResNet18 · CIFAR-100 | test accuracy: 66.3% -> 78.3% | Improved |
| raise learning rate from 0.05 to 0.2 (ablation) | ResNet18 · CIFAR-100 | test accuracy: 78.3% -> 74.5% | Hurt |
| use ResNet18 instead of MLP (ablation) | ResNet18 · CIFAR-10 | test accuracy: 55.4% -> 94.8% | Improved |
| train the full network instead of last layer only (ablation) | CNN · MNIST | D_LR: 0.51 -> 0.043 | Improved |
| shuffle 10% of labels (ablation) | ResNet18 · CIFAR-10 | test accuracy: 95.0% -> 66.1% | Hurt |
| shuffle labels fully (ablation) | ResNet18 · CIFAR-10 | D_LR: 0.031 -> 0.034 | No clear effect |
| train through grokking from step 200 to step 3000 | 2-layer transformer · Mod 97 | D_LR: 0.51 -> 0.11 | Improved |
| increase target downscaling alpha from 0.2 to 20 (ablation) | MLP · CIFAR-10 | test accuracy: 92.7% -> 88.3% | Hurt |
Closest papers by meaning
- From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks (ICLR 2025 · similarity 0.86)
- Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning (NeurIPS 2026 · similarity 0.86)
- Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning (NeurIPS 2024 · similarity 0.85)
- Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.85)
- Metric Space Magnitude for Evaluating the Diversity of Latent Representations (NeurIPS 2024 · similarity 0.85)
- Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
