Research map / Learning theory and algorithms / Sample complexity
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a convolutional architecture rather than an orthogonal-equivariant fully-connected training algorithm | Quadratic classification on Gaussian N(0,I_{2d}) · Gaussian N(0,I_{2d}), fixed quadratic-label task | sample complexity: \Omega(d^2) vs O\left(\frac{1}{{\epsilon}}\left( \log\frac{1}{{\epsilon}}+ \log\frac{1}{\delta}\right)\right) | Helped |
| Use a convolutional architecture rather than an orthogonal-equivariant fully-connected training algorithm | Quadratic threshold class on N(0,I_d) · N(0,I_d), quadratic threshold class | sample complexity: \Omega(d^2/{\epsilon}) vs O(\frac{d\log\frac{1}{{\epsilon}}+\log\frac{1}{\delta}}{{\epsilon}}) | Helped |
| Use a convolutional architecture rather than an orthogonal-equivariant fully-connected training algorithm | Quadratic regression class on N(0,I_d) · N(0,I_d), quadratic regression class | \ell_2 sample complexity: \ge \frac{d(d+3)}{2}(1-{\epsilon}) -1 vs \le d | Helped |
| Use a convolutional architecture rather than a permutation-equivariant fully-connected training algorithm | Local texture classification · Local texture classification, 1-dimensional image | sample complexity: \ge \frac{d}{10} vs \le \log_2{\frac{1}{\delta}}+2 | Helped |
Closest papers by meaning
- How Many Samples are Needed to Estimate a Convolutional Neural Network? (NeurIPS 2018 · similarity 0.88)
- The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes (NeurIPS 2022 · similarity 0.87)
- Computational Separation Between Convolutional and Fully-Connected Networks (ICLR 2021 · similarity 0.87)
- Approximation and non-parametric estimation of ResNet-type convolutional neural networks (ICML 2019 · similarity 0.87)
- Inductive Bias of Deep Convolutional Networks through Pooling Geometry (ICLR 2017 · similarity 0.86)
- Sampling weights of deep neural networks (NeurIPS 2023 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.