Research map / Architectures and efficiency / CNN design
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Joan Bruna
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize FCN-space training from an untrained CNN embedding rather than standard FCN initialization | eFCN · CIFAR-10 | test accuracy: approximately at 62.5%; standard FCN result of 57.5% | Improved |
| Relax CNN constraints at an intermediate training time (ablation) | eFCN · CIFAR-10 | test accuracy | Improved |
| Relax CNN constraints only after late CNN training (ablation) | eFCN · CIFAR-10 | test accuracy | No clear effect |
| Train the eFCN after relaxing its constraints (ablation) | eFCN · CIFAR-10 | test accuracy | Improved |
| Use off-local weights without local weights (ablation) | eFCN · CIFAR-10 | test accuracy | No clear effect |
| Add off-local weights alongside local weights (ablation) | eFCN · CIFAR-10 | test accuracy | Improved |
| Interpolate between the trained CNN and eFCN in weight space (ablation) | CNN and eFCN solutions · CIFAR-10 | test accuracy | Improved |
| Use output-space interpolation instead of weight-space interpolation (ablation) | CNN and eFCN solutions · CIFAR-10 | test accuracy gain | Improved |
| Use Adam instead of SGD (ablation) | VanillaCNN and eFCN · CIFAR-10 | test accuracy | Mixed |
Closest papers by meaning
- Linear CNNs Discover the Statistical Structure of the Dataset Using Only the Most Dominant Frequencies (ICML 2023 · similarity 0.87)
- An Empirical Evaluation of Current Convolutional Architectures' Ability to Manage Nuisance Location and Scale Variability (CVPR 2016 · similarity 0.86)
- Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs (ICLR 2024 · similarity 0.85)
- Initial Guessing Bias: How Untrained Networks Favor Some Classes (ICML 2024 · similarity 0.84)
- Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild (CVPR 2025 · similarity 0.84)
- ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
