Research map / Architectures and efficiency / CNN design
Computational Separation Between Convolutional and Fully-Connected Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a CNN with gradient descent and local receptive fields | Learning k-patterns · k-pattern problem | learnability in polynomial time: q = O(\epsilon^{-2} n^3 \log^2 n); T = O(\epsilon^{-2} n^3 \log n) | Helped |
| Use a fully-connected network with permutation-invariant initialization instead of a locality-based architecture | Learning k-patterns · Uniform input distribution; k = Ω(log n) | initial gradient norm: qn^{-\Omega(\log n)} | Hurt |
| Remove weight sharing while retaining local receptive fields | CNN · k-pattern problem | learnability | No clear effect |
| Use CNN or LCN locality for consecutive-digit parity | MNIST sequence classification · MNIST sequences, consecutive parity of 3 middle digits | classification performance | Helped |
| Use an FCN rather than a CNN or LCN when relevant digits are far apart | MNIST sequence classification · MNIST sequences, parity of first, middle, and last digits | classification performance | Helped |
| Break the local structure by placing the relevant digits far apart | CNN and LCN · MNIST sequences, parity of first, middle, and last digits | classification performance | Hurt |
| Increase sequence length for the consecutive-digit parity task | FCN · MNIST sequences, consecutive parity of 3 middle digits | classification performance: n=19 | Hurt |
Closest papers by meaning
- The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes (NeurIPS 2022 · similarity 0.88)
- Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? (ICLR 2021 · similarity 0.87)
- Theoretical Analysis of the Inductive Biases in Deep Convolutional Networks (NeurIPS 2023 · similarity 0.86)
- Towards Biologically Plausible Convolutional Networks (NeurIPS 2021 · similarity 0.86)
- Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs (ICLR 2024 · similarity 0.86)
- Approximation and Learning with Deep Convolutional Models: a Kernel Perspective (ICLR 2022 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.