Research map / Optimization / Implicit bias and generalization
Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use weight-tied orthogonal encoder and decoder matrices | Two-layer autoencoder · Isotropic Gaussian, r ≤ 1 | population risk | Helped |
| Use weight-tied rotationally invariant matrices with the specified spectrum | Two-layer autoencoder · Isotropic Gaussian, r > 1, high-dimensional limit | population risk | Helped |
| Optimize with weight-tied gradient flow from a full-rank initialization | Two-layer autoencoder · Isotropic Gaussian, r ≤ 1 | population risk | Helped |
| Use projected gradient descent with row-normalized Gaussian initialization and step size \eta=\Theta(1/\sqrt{d}) | Two-layer autoencoder · Isotropic Gaussian, r < 1, sufficiently large d | population risk | Helped |
| Use a two-layer sign autoencoder rather than the information-theoretically optimal general encoder/decoder | Sign autoencoder · Gaussian compression, all r > 0 | reconstruction error | Hurt |
| Use a two-layer sign autoencoder rather than VAMP | Sign autoencoder · Isotropic Gaussian compression, r > 1 | reconstruction error | Hurt |
| Use a two-layer sign autoencoder with a rotationally invariant fixed encoder | Sign autoencoder · Isotropic Gaussian compression, r ≤ 1 | reconstruction error | Helped |
| Use the Gaussian-theory predictions on standard datasets | Two-layer autoencoder · CIFAR-10 and MNIST | population risk agreement | Helped |
Closest papers by meaning
- Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth (ICML 2024 · similarity 0.90)
- Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure (ICLR 2025 · similarity 0.86)
- Asymptotics of feature learning in two-layer networks after one gradient-step (ICML 2024 · similarity 0.85)
- Scaling and evaluating sparse autoencoders (ICLR 2025 · similarity 0.85)
- Low Rank Gradients and Where to Find Them (NeurIPS 2025 · similarity 0.84)
- Auto-Compressing Networks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.