Research map / Label-efficient and robust learning / Recognition and long-tail
FACT: a first-principles alternative to the Neural Feature Ansatz for how networks learn representations
Enric Boix-Adserà, Neil Rohit Mallinar, James B Simon, Mikhail Belkin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FACT-RFM update instead of NFA-RFM (no geometric averaging) | RFM · UCI tabular (120 datasets) | average test accuracy (%): 85.10 -> 85.22 | No clear effect |
| add geometric averaging to FACT-RFM (ablation) | FACT-RFM · UCI tabular (120 datasets) | average test accuracy (%): 85.22 -> 84.99 | No clear effect |
| add feature learning (FACT-RFM) to Laplace kernel regression | kernel regression · UCI tabular (120 datasets) | average test accuracy (%): 83.71 -> 85.22 | Improved |
| use FACT instead of sqrt(AGOP) as predictor of W^T W | 5-layer ReLU MLP · MNIST (n=60K) | Pearson correlation: 0.909 -> 0.999 | Improved |
| use FACT instead of sqrt(AGOP) on CIFAR-10 at weight decay 1e-4 | 5-layer ReLU MLP · CIFAR-10 (n=50K) | Pearson correlation: 0.722 -> 0.896 | Improved |
| use FACT instead of sqrt(AGOP) at n=1K | 5-layer ReLU MLP · MNIST (n=1K) | Pearson correlation: 0.899 -> 0.890 | No clear effect |
| symmetrize FACT (sqrt(FACT FACT^T)) (ablation) | FACT · MNIST (n=1K) | Pearson correlation: 0.890 -> 0.956 | Improved |
| increase weight decay from 1e-4 to 1e-3 on CIFAR-10 (ablation) | FACT · CIFAR-10 (n=50K) | Pearson correlation: 0.896 -> 0.830 | Hurt |
| increase weight decay from 1e-4 to 1e-3 on CIFAR-10 (ablation) | sqrt(AGOP) · CIFAR-10 (n=50K) | Pearson correlation: 0.722 -> 0.793 | Improved |
| increase MNIST training set size from 1K to 60K (ablation) | FACT · MNIST | Pearson correlation: 0.890 -> 0.999 | Improved |
| construct synthetic data where NFA fails: FACT vs AGOP | two-layer quadratic network · Synthetic quadratic network (width 10) | cosine similarity: 0.068 -> 0.994 | Improved |
| replace NFA update with FACT update on modular addition | RFM · Modular addition p=61 | test accuracy (%): 100% -> 100% | No clear effect |
Closest papers by meaning
- Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks (NeurIPS 2026 · similarity 0.86)
- Feature emergence via margin maximization: case studies in algebraic tasks (ICLR 2024 · similarity 0.86)
- Complexity Matters: Feature Learning in the Presence of Spurious Correlations (ICML 2024 · similarity 0.86)
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.85)
- How are linear representations learned? Exact solutions to the dynamics of abstraction (NeurIPS 2026 · similarity 0.85)
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
