AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Neural Functional Transformers

Allan Zhou, Kaien Yang, Yiding Jiang, Kaylee Burns, Winnie Xu, Samuel Sokota, J Zico Kolter, Chelsea Finn

NeurIPS 2023 · Architectures and efficiency › RNNs and normalization · approach: Mechanistic circuit interpretability · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace NFN weight-space layers with attention-based NFT layersINR editing · MNIST erosion, dilation, and gradient; FashionMNIST gradient; CIFAR contrasttest MSE: 0.0223 ± 0.0000 -> 0.0194 ± 0.0002; 0.0693 ± 0.0009 -> 0.0510 ± 0.0004; 0.0566±0.0000 -> 0.0484±0.0007; 0.0870 ± 0.0001 Improved
use NFT rather than NFN_HNP for CNN generalization predictionCNN generalization predictor · CIFAR-10-GSKendall rank correlation τ: 0.934 ± 0.001 -> 0.926 ± 0.001Hurt
use NFT rather than NFN_HNP for CNN generalization predictionCNN generalization predictor · SVHN-GSKendall rank correlation τ: 0.931 ± 0.005 -> 0.858 ± 0.000Hurt
replace the NFN encoder with an NFT encoder (ablation)Inr2Array · MNIST INR classificationtest accuracy (%): 94.6 ± 0.00 -> 98.5 ± 0.00Improved
replace the NFN encoder with an NFT encoder (ablation)Inr2Array · FashionMNIST INR classificationtest accuracy (%): 76.7 ± 0.00 -> 79.3 ± 0.00Improved
replace the NFN encoder with an NFT encoder (ablation)Inr2Array · CIFAR-10 INR classificationtest accuracy (%): 45.4 ± 0.00 -> 63.4 ± 0.00Improved
add spatial activations without Inr2Array reconstruction training (ablation)NFN INR classifier · MNIST, FashionMNIST, and CIFAR-10 INR classificationtest accuracy (%): 92.9±0.46 vs 92.9 ± 0.38; 70.8±0.53 vs 75.6 ± 1.07; 45.6 ± 0.11 vs 46.6 ± 0.13Hurt
replace the Inr2Array NFT encoder with an NFN encoder (ablation)Inr2Array reconstruction · MNIST, FashionMNIST, and CIFAR-10test MSE: 0.027 ± 0.003 -> 0.046 ± 0.001; 0.070 ± 0.006 -> 0.085 ± 0.006; 0.036 ± 0.006 -> 0.117 ± 0.008Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy