Research map / Architectures and efficiency / RNNs and normalization
Emergence of Invariance and Disentanglement in Deep Representations
Alessandro Achille, Stefano Soatto
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| raise information-bottleneck weight beta above 1 when training on random labels (ablation) | Small AlexNet, AllCNN, ResNet with IB Lagrangian · CIFAR-10 random labels (10000 samples) | training fit / memorization | Improved |
| increase fraction of corrupted labels (ablation) | IB Lagrangian network at beta=0.1 · CIFAR-10 label corruption (30000 samples) | information in weights I(w;D): increases by ~3 nats/sample for completely random labels vs 2.30 nats/sample for 10-class labels | Hurt |
| decrease information in the weights by increasing beta (ablation) | MNIST digit classifier with Gaussian multiplicative weight noise · Cluttered MNIST | nuisance information I(z;n) | Improved |
Closest papers by meaning
- In What Ways Are Deep Neural Networks Invariant and How Should We Measure This? (NeurIPS 2022 · similarity 0.90)
- Cross-Entropy Is All You Need To Invert the Data Generating Process (ICLR 2025 · similarity 0.88)
- Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off (ICLR 2025 · similarity 0.87)
- From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication (ICLR 2024 · similarity 0.87)
- $\alpha$TC-VAE: On the relationship between Disentanglement and Diversity (ICLR 2024 · similarity 0.87)
- From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
