AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning

Hien Dang, Pratik Patil, Alessandro Rinaldo

ICML 2026 · Learning theory and algorithms › Kernels and regression theory · approach: Kernel ridge regression analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add optimally mixed self-distillation (unconstrained xi) to the ridge teacherRidge regression · BlogFeedback, Communities and Crime, ResNet-18 CIFAR10 featurestest squared prediction riskImproved
Allow negative mixing weight xi in over-regularized regimes (ablation)Self-distilled ridge · Real-data ridge pathtest squared prediction riskImproved
Add self-distillation to a ridge teacher under distribution shiftRidge regression · Air Quality (out-of-distribution)test squared prediction riskImproved
Add optimal self-distillation to a ridge teacher at the globally optimal lambdaRidge regression · Communities and Crime, CIFAR10 featuresglobal minimum risk over lambdaNo clear effect
Add optimal self-distillation to a ridge teacher at the globally optimal lambdaRidge regression · BlogFeedback, Air Qualityglobal minimum risk over lambdaImproved
Use pure-distillation (xi=1) instead of optimal mixing (ablation)Ridge regression · Real-data ridge pathtest squared prediction riskHurt
Restrict xi to [0,1] instead of unconstrained (ablation)Self-distilled ridge · Over-regularized teachertest squared prediction riskHurt
Use recursive multi-round self-distillation with optimized xi each roundSelf-distilled ridge · Air Quality, Communities and Crimetest squared prediction riskImproved
Anchor later rounds to the original labels instead of recursive mixing (ablation)Multi-round self-distillation · Air Quality, Communities and Crimetest squared prediction riskHurt
Distill on fresh unlabeled features instead of the same training features (ablation)Self-distilled ridge · Isotropic setting n=200, p=100test squared prediction riskHurt
Estimate xi with one-shot GCV instead of the oracleSelf-distilled ridge · CIFAR10/CIFAR100 ResNet features, UCI dataSD risk estimateImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy