AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

A Functional Perspective on Learning Symmetric Functions with Neural Networks

Aaron Zweig, Joan Bruna

ICML 2021 · Learning theory and algorithms › Kernels and regression theory · approach: Generalization theory of deep networks · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Train the first two layers as well as the final layer (S1 instead of S3) (ablation)Measure networks · Symmetric function approximation, planted neurontest errorImproved
Train the first layer rather than freeze it (S1 instead of S2) (ablation)Measure networks · Symmetric function approximation, planted neurontest errorImproved
Use random kernel features that represent the smooth neuron (S2 rather than S1) (ablation)Measure networks · Symmetric function approximation, smooth neurontest errorNo clear effect
Freeze the second layer as well (S3 rather than S2) (ablation)Measure networks · Symmetric function approximation, smooth neurontest errorHurt
Increase the number of random kernel features m (ablation)S2 and S3 · Symmetric function approximation, planted neurontest errorMixed
Normalize the pooled set representation by taking an average instead of a sum (ablation)DeepSets architecture · Out-of-distribution input-size generalizationtest errorImproved
Train on set size N = 20 with S1 rather than use the sample meanRobust mean estimation · Robust mean estimation, N = 20mean squared test error: 0.153 ± 0.068 -> 0.131 ± 0.018Improved
Train on set size N = 20 with S1 rather than use the adversarial estimatorRobust mean estimation · Robust mean estimation, N = 20mean squared test error: 0.469 ± 0.550 -> 0.131 ± 0.018Improved
Use S1 rather than S2 (ablation)Robust mean estimation · Robust mean estimation, N = 20mean squared test error: 0.137 ± 0.019 -> 0.131 ± 0.018No clear effect
Use S1 rather than S3 (ablation)Robust mean estimation · Robust mean estimation, N = 20mean squared test error: 0.144 ± 0.020 -> 0.131 ± 0.018Improved
Train on fixed-size sets and evaluate across different input sizesMeasure networks · Second-largest-element functiontest errorHurt
Approximate max with the softmax family rather than use max directlyMeasure networks · Out-of-distribution input-size generalizationtest errorImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy