Research map / Optimization / Optimizers
Learning High-Dimensional Parity Functions with Product Networks using Gradient Descent
Guillaume Larue, Louis-Adrien Dufrène, Quentin Lampin, Hadi Ghauch, Rekaya-Ben Othman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use product XOR node instead of MLP when training on unit-sparse inputs | parity learner · Subset parity N=16, sparse regime | validation accuracy on full truth table: MLP stagnates near 50% vs product node 100% | Improved |
| train on uniform samples of the truth table instead of sparse samples (ablation) | Product node vs MLP · Subset parity N=16, uniform regime | convergence speed | Mixed |
| use constant pe=2/N instead of the theoretical pe=1/N (ablation) | Product node · Subset parity N=100 | training steps to convergence: optimum p*e about 2/N | Improved |
| use dense inputs from the start (pe at or above about 7/N) (ablation) | Product node · Subset parity N=100 | convergence within 25,000 steps: no convergence for pe >= 7/N | Hurt |
| use pe far below 1/N (ultra-sparse inputs) (ablation) | Product node · Subset parity N=100 | convergence speed | Hurt |
| raise learning rate up to about N/2 (ablation) | Product node · Subset parity N from 10 to 1000 | convergence time: alpha_lim about N/2 | Improved |
| raise learning rate beyond alpha_lim (ablation) | Product node · Subset parity N from 10 to 1000 | convergence | Hurt |
| vary the proportion pw of oracle weights equal to 1 (ablation) | Product node · Subset parity N=100,000 | convergence | No clear effect |
| scale learning rate with N to keep alpha times pe constant (ablation) | Product node · Subset parity across N | convergence speed | Improved |
Closest papers by meaning
- Learning High-Degree Parities: The Crucial Role of the Initialization (ICLR 2025 · similarity 0.87)
- Matching the Statistical Query Lower Bound for $k$-Sparse Parity Problems with Sign Stochastic Gradient Descent (NeurIPS 2024 · similarity 0.85)
- Transformers Provably Solve Parity Efficiently with Chain of Thought (ICLR 2025 · similarity 0.85)
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.85)
- On the Complexity of Learning Sparse Functions with Statistical and Gradient Queries (NeurIPS 2024 · similarity 0.84)
- Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
