AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

What Do Neural Networks Learn When Trained With Random Labels?

Hartmut Maennel, Ibrahim M Alabdulmohsin, Ilya O Tolstikhin, Robert Baldock, Olivier Bousquet, Sylvain Gelly, Daniel Keysers

NeurIPS 2020 · Architectures and efficiency › Pruning and efficient tuning · approach: Parameter-efficient transfer analysis · 32 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add sampled convolutional layer 1 to the initialization (ablation)Simple CNN · CIFAR10, 100 iterationstrain accuracy: 0.31 -> 0.34Improved
Add sampled convolutional layer 2 (ablation)Simple CNN with layer 1 sampled · CIFAR10, 100 iterationstrain accuracy: 0.34 -> 0.38Improved
Add sampled convolutional layer 3 (ablation)Simple CNN with layers 1–2 sampled · CIFAR10, 100 iterationstrain accuracy: 0.38 -> 0.41Improved
Add sampled convolutional layers 1–3 rather than sampling none (ablation)Simple CNN · CIFAR10, 100 iterationstest accuracy: 0.31 -> 0.40Improved
Add sampled convolutional layer 1 to the initialization (ablation)Simple CNN · CIFAR10, 1000 iterationstrain accuracy: 0.58 -> 0.61Improved
Add sampled convolutional layer 2 (ablation)Simple CNN with layer 1 sampled · CIFAR10, 1000 iterationstrain accuracy: 0.61 -> 0.67Improved
Add sampled convolutional layer 3 (ablation)Simple CNN with layers 1–2 sampled · CIFAR10, 1000 iterationstrain accuracy: 0.67 -> 0.68Improved
Add sampled convolutional layer 1 to the initialization (ablation)Simple CNN · CIFAR10, 1000 iterationstest accuracy: 0.53 -> 0.55Improved
Add sampled convolutional layer 2 (ablation)Simple CNN with layer 1 sampled · CIFAR10, 1000 iterationstest accuracy: 0.55 -> 0.56Improved
Add sampled convolutional layer 3 (ablation)Simple CNN with layers 1–2 sampled · CIFAR10, 1000 iterationstest accuracy: 0.56 -> 0.56No clear effect
Pre-train on random labels before downstream training, while matching weight norms to initializationVGG16 and ResNet18 · CIFAR10 and ImageNet downstream taskstraining speedImproved
Initialize first-layer filters from the covariance learned during random-label training instead of using the pre-trained filtersSimple CNN · CIFAR10 downstream taskstraining speedImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy