Research map / Learning theory and algorithms / Kernels and regression theory
A Functional Perspective on Learning Symmetric Functions with Neural Networks
Aaron Zweig, Joan Bruna
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the first two layers as well as the final layer (S1 instead of S3) (ablation) | Measure networks · Symmetric function approximation, planted neuron | test error | Improved |
| Train the first layer rather than freeze it (S1 instead of S2) (ablation) | Measure networks · Symmetric function approximation, planted neuron | test error | Improved |
| Use random kernel features that represent the smooth neuron (S2 rather than S1) (ablation) | Measure networks · Symmetric function approximation, smooth neuron | test error | No clear effect |
| Freeze the second layer as well (S3 rather than S2) (ablation) | Measure networks · Symmetric function approximation, smooth neuron | test error | Hurt |
| Increase the number of random kernel features m (ablation) | S2 and S3 · Symmetric function approximation, planted neuron | test error | Mixed |
| Normalize the pooled set representation by taking an average instead of a sum (ablation) | DeepSets architecture · Out-of-distribution input-size generalization | test error | Improved |
| Train on set size N = 20 with S1 rather than use the sample mean | Robust mean estimation · Robust mean estimation, N = 20 | mean squared test error: 0.153 ± 0.068 -> 0.131 ± 0.018 | Improved |
| Train on set size N = 20 with S1 rather than use the adversarial estimator | Robust mean estimation · Robust mean estimation, N = 20 | mean squared test error: 0.469 ± 0.550 -> 0.131 ± 0.018 | Improved |
| Use S1 rather than S2 (ablation) | Robust mean estimation · Robust mean estimation, N = 20 | mean squared test error: 0.137 ± 0.019 -> 0.131 ± 0.018 | No clear effect |
| Use S1 rather than S3 (ablation) | Robust mean estimation · Robust mean estimation, N = 20 | mean squared test error: 0.144 ± 0.020 -> 0.131 ± 0.018 | Improved |
| Train on fixed-size sets and evaluate across different input sizes | Measure networks · Second-largest-element function | test error | Hurt |
| Approximate max with the softmax family rather than use max directly | Measure networks · Out-of-distribution input-size generalization | test error | Improved |
Closest papers by meaning
- How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning (ICLR 2025 · similarity 0.88)
- Symmetries in Overparametrized Neural Networks: A Mean Field View (NeurIPS 2024 · similarity 0.86)
- Function-space Parameterization of Neural Networks for Sequential Learning (ICLR 2024 · similarity 0.86)
- Generalization Bounds and Model Complexity for Kolmogorov–Arnold Networks (ICLR 2025 · similarity 0.86)
- Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples (ICML 2024 · similarity 0.86)
- Symmetric Single Index Learning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
