AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Analyzing Generalization of Neural Networks through Loss Path Kernels

Yilan Chen, Wei Huang, Hao Wang, Charlotte Loh, Akash Srivastava, Lam M. Nguyen, Tsui-Wei Weng

NeurIPS 2023 · Optimization › Implicit bias and generalization · approach: Stability-based generalization analysis

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use Gene(w, S) after 1 epoch instead of TENAS to pick architecturesrandom search NAS · NAS-Bench-201 CIFAR-10test accuracy: 93.08±0.15 -> 93.68±0.12Improved
Use Gene(w, S) after 2 epochs instead of TENASrandom search NAS · NAS-Bench-201 CIFAR-10test accuracy: 93.08±0.15 -> 93.79±0.18Improved
Use Gene(w, S) after 1 epoch instead of TENASrandom search NAS · NAS-Bench-201 CIFAR-100test accuracy: 70.37±2.40 -> 72.02±1.43Improved
Use Gene(w, S) after 2 epochs instead of TENASrandom search NAS · NAS-Bench-201 CIFAR-100test accuracy: 70.37±2.40 -> 72.76±0.33Improved
Use Gene(w, S) after 1 epoch instead of RS + LGA3random search NAS · NAS-Bench-201 CIFAR-100test accuracy: 69.77 -> 72.02±1.43Improved
Use Gene(w, S) after 2 epochs (vs 1 epoch) (ablation)RS + Gene(w, S) · NAS-Bench-201 CIFAR-10test accuracy: 93.68±0.12 -> 93.79±0.18Improved
Use Gene(w, S) after 2 epochs (vs 1 epoch) (ablation)RS + Gene(w, S) · NAS-Bench-201 CIFAR-100best-run accuracy: 73.15 -> 73.15No clear effect
Compare Gene(w, S) search with training each architecture to convergenceRS + Gene(w, S) · NAS-Bench-201 CIFAR-10GPU hours per architecture: 1.83 -> 0.036Improved
Compare Gene(w, S) search with training each architecture to convergenceRS + Gene(w, S) · NAS-Bench-201 CIFAR-100GPU hours per architecture: 2.56 -> 0.037Improved
Replace norm-based bound with loss path kernel boundgeneralization bound · MNIST 1 vs 7, two-layer NNbound value: 140.7 -> 0.03Improved
Replace VC dimension bound with loss path kernel boundgeneralization bound · MNIST 1 vs 7, two-layer NNbound value: 55957.3 -> 0.03Improved
Replace Cao & Gu NTK-based bound with loss path kernel boundgeneralization bound · MNIST 1 vs 7, 1000-hidden NNbound value: 1.44 -> 0.026Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy