AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

FACT: a first-principles alternative to the Neural Feature Ansatz for how networks learn representations

Enric Boix-Adserà, Neil Rohit Mallinar, James B Simon, Mikhail Belkin

ICLR 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Contrastive representation theory

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use FACT-RFM update instead of NFA-RFM (no geometric averaging)RFM · UCI tabular (120 datasets)average test accuracy (%): 85.10 -> 85.22No clear effect
add geometric averaging to FACT-RFM (ablation)FACT-RFM · UCI tabular (120 datasets)average test accuracy (%): 85.22 -> 84.99No clear effect
add feature learning (FACT-RFM) to Laplace kernel regressionkernel regression · UCI tabular (120 datasets)average test accuracy (%): 83.71 -> 85.22Improved
use FACT instead of sqrt(AGOP) as predictor of W^T W5-layer ReLU MLP · MNIST (n=60K)Pearson correlation: 0.909 -> 0.999Improved
use FACT instead of sqrt(AGOP) on CIFAR-10 at weight decay 1e-45-layer ReLU MLP · CIFAR-10 (n=50K)Pearson correlation: 0.722 -> 0.896Improved
use FACT instead of sqrt(AGOP) at n=1K5-layer ReLU MLP · MNIST (n=1K)Pearson correlation: 0.899 -> 0.890No clear effect
symmetrize FACT (sqrt(FACT FACT^T)) (ablation)FACT · MNIST (n=1K)Pearson correlation: 0.890 -> 0.956Improved
increase weight decay from 1e-4 to 1e-3 on CIFAR-10 (ablation)FACT · CIFAR-10 (n=50K)Pearson correlation: 0.896 -> 0.830Hurt
increase weight decay from 1e-4 to 1e-3 on CIFAR-10 (ablation)sqrt(AGOP) · CIFAR-10 (n=50K)Pearson correlation: 0.722 -> 0.793Improved
increase MNIST training set size from 1K to 60K (ablation)FACT · MNISTPearson correlation: 0.890 -> 0.999Improved
construct synthetic data where NFA fails: FACT vs AGOPtwo-layer quadratic network · Synthetic quadratic network (width 10)cosine similarity: 0.068 -> 0.994Improved
replace NFA update with FACT update on modular additionRFM · Modular addition p=61test accuracy (%): 100% -> 100%No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy