AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

Escaping Parameter Space: Tight Generalization Bounds via Representation Quality

Niclas A Göring, Shuofeng Zhang, Branton DeMoss, Ard Louis

NeurIPS 2026 · Learning theory and algorithms › Kernels and regression theory · approach: Information-theoretic generalization bounds · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives a tight generalization bound for deterministic neural networks from last-hidden-layer representation quality, evaluated with a parameter-free k-NN classifier.

Problem

Generalization bounds for deterministic neural networks are typically loose because they rely on parameter-space complexity measures, posterior KL divergences, weight norms, or compression arguments.

Approach

Bounds generalization directly from the quality of the last hidden layer representations, using a parameter-free k-nearest-neighbor classifier as the evaluator, thereby avoiding parameter-space complexity measures.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
certify via a k-NN decoder on a pretrained representation instead of random init (ablation)ResNet-18 transfer certificate · CIFAR-10certified bound (%): 65.8 -> 20.1Improved
scale the pretrained model from DINOv2-L (304M) to DINOv2-g (1.1B) (ablation)k-NN transfer certificate · ImageNet valcertified bound (%): 23.1 -> 23.5No clear effect
use pretrained ViT representation instead of training ViT from scratch (ablation)ViT-small certificate · CIFAR-100certified bound (%): 99.3 -> 30.5Improved
switch ReLU to SiLU activation (ablation)ResNet-18 cross-fit certificate · CIFAR-100certified bound (%): 51.1 -> 58.6Hurt
switch ReLU to SiLU activation (ablation)FFNN cross-fit certificate · MNISTcertified bound (%): 4.9 -> 4.8No clear effect
switch ReLU to SiLU activation (ablation)ViT cross-fit certificate · CIFAR-10certified bound (%): 31.9 -> 32.3No clear effect
use ResNet-50 instead of ResNet-18 (ablation)cross-fit certificate (ReLU) · CIFAR-100certified bound (%): 51.1 -> 44.6Improved
train a small ViT from scratch on CIFAR-100 instead of a ResNet (ablation)cross-fit certificate · CIFAR-100certified bound (%): 44.6 -> 99.3Hurt
add a regularizer pushing same-class representations apart (destroy local class geometry) (ablation)six-layer FFNN on SVHN · SVHNtest accuracyHurt
increase network width in ResNet-18 (ablation)local class geometry term Q1 · CIFAR-10 width sweepQ1 excess over widest networkImproved
increase width through the interpolation threshold (ablation)decoder-network agreement term Q2 · CIFAR-10 width sweepQ2 excessMixed
certify through representation rather than through weight noisek-NN decoder vs network · CIFAR-10, Gaussian weight noise sigma 0.01accuracy (%): 67 -> 84Improved

Experimental setup

Metrics: generalization bound tightness test error

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy