Research map / Label-efficient and robust learning / Knowledge distillation
Learning curves of generic features maps for realistic datasets with a teacher-student model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the Gaussian covariate model to predict learning curves for ridge regression with linear teachers on real data | Ridge regression on Fashion-MNIST · Fashion-MNIST | training and test error | Helped |
| Use the Gaussian covariate model to predict learning curves for random features and scattering features | Ridge regression with linear teachers · MNIST | training and test error | Helped |
| Apply the Gaussian covariate model to binary classification on real data | Classification with real data features · Fashion-MNIST | classification error | Hurt |
| Apply the Gaussian covariate model to logistic regression on real rather than GAN-generated CIFAR10 images | Logistic regression · CIFAR10 | learning curve prediction | Hurt |
| Use features at epoch 0 instead of early learned features | Student network trained on GAN-generated CIFAR10-like images · CIFAR10-like GAN data | classification performance | Helped |
| Train the student features to epoch 200 rather than epoch 50 | Student network trained on GAN-generated CIFAR10-like images · CIFAR10-like GAN data | sample complexity for separability | Hurt |
| Train the student features to epoch 200 rather than epoch 50 | Student network trained on GAN-generated CIFAR10-like images · CIFAR10-like GAN data | training and generalisation performance | Helped |
| Use the Gaussian covariate prediction closer to the full real-data sample size | Ridge regression with linear teachers · Fashion-MNIST and MNIST | learning curve prediction | Hurt |
Closest papers by meaning
- High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation (NeurIPS 2022 · similarity 0.86)
- Asymptotics of Learning with Deep Structured (Random) Features (ICML 2024 · similarity 0.86)
- LCDB 1.1: A Database Illustrating Learning Curves Are More Ill-Behaved Than Previously Thought (NeurIPS 2025 · similarity 0.85)
- Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure (ICLR 2025 · similarity 0.85)
- Weak-to-Strong Generalization Even in Random Feature Networks, Provably (ICML 2025 · similarity 0.85)
- Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.